-- Logs begin at Wed 2022-07-20 08:34:32 UTC, end at Wed 2022-07-20 09:35:13 UTC. -- Jul 20 08:58:26.492282 dv-cinder-843309-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 20 08:58:26.500343 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 20 08:58:26.502504 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 20 08:58:26 2022] *** Jul 20 08:58:26.502504 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 20 08:58:26.502504 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 20 08:58:26.502504 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: nodename: dv-cinder-843309-5-iscsi Jul 20 08:58:26.502504 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: machine: x86_64 Jul 20 08:58:26.502504 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: clock source: unix Jul 20 08:58:26.502504 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: pcre jit disabled Jul 20 08:58:26.502504 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: detected number of CPU cores: 4 Jul 20 08:58:26.502504 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: current working directory: / Jul 20 08:58:26.502504 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: detected binary path: /usr/bin/uwsgi-core Jul 20 08:58:26.503072 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: your processes number limit is 47680 Jul 20 08:58:26.503072 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: your memory page size is 4096 bytes Jul 20 08:58:26.503072 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: detected max file descriptor number: 2048 Jul 20 08:58:26.503072 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: lock engine: pthread robust mutexes Jul 20 08:58:26.503072 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: thunder lock: enabled Jul 20 08:58:26.503072 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 20 08:58:26.503072 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 20 08:58:26.547009 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: Python main interpreter initialized at 0x555ab68dc9c0 Jul 20 08:58:26.547009 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: python threads support enabled Jul 20 08:58:26.547009 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: your server socket listen backlog is limited to 100 connections Jul 20 08:58:26.547009 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: your mercy for graceful operations on workers is 90 seconds Jul 20 08:58:26.547354 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: mapped 403077 bytes (393 KB) for 2 cores Jul 20 08:58:26.547717 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: *** Operational MODE: preforking *** Jul 20 08:58:26.547999 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: *** uWSGI is running in multiple interpreter mode *** Jul 20 08:58:26.547999 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: spawned uWSGI master process (pid: 80329) Jul 20 08:58:26.548390 dv-cinder-843309-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 20 08:58:26.549273 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: spawned uWSGI worker 1 (pid: 80333, cores: 1) Jul 20 08:58:26.549273 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: spawned uWSGI worker 2 (pid: 80334, cores: 1) Jul 20 08:58:26.549273 dv-cinder-843309-5-iscsi devstack@keystone.service[80329]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 20 08:58:32.040246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 08:58:32.040246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: from cryptography.utils import int_from_bytes Jul 20 08:58:32.041102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 08:58:32.041102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: from cryptography.utils import int_from_bytes Jul 20 08:58:32.042175 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 08:58:32.042175 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: from cryptography.utils import int_from_bytes Jul 20 08:58:32.042623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 08:58:32.042623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: from cryptography.utils import int_from_bytes Jul 20 08:58:32.090201 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 08:58:32.090407 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 08:58:32.090623 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] command line args: [] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 08:58:32.090933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 08:58:32.091132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ================================================================================ {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 08:58:32.091395 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] admin_token = **** {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.091660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] config_dir = [] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.091764 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 08:58:32.091870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.091977 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 08:58:32.092061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] config_source = [] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.092175 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] command line args: [] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 08:58:32.092262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] debug = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.092442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.092682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 08:58:32.092682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ================================================================================ {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 08:58:32.092805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.092865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] admin_token = **** {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.093003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.093148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] config_dir = [] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.093273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.093387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.093531 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.093653 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] config_source = [] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.093771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] debug = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.093881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] list_limit = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.093997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.094146 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] log_config_append = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.094146 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.094368 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.095038 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] log_dir = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.095207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.095332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] log_file = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.095506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.095630 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.095750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.095858 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.095952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] list_limit = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.096051 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.096153 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] log_config_append = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.096272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.096392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.096503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.096585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] log_dir = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.096705 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.096788 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] log_file = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.096859 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.096926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.097039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.097118 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.097195 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.097272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.097395 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.097541 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.097624 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.097723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.097905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.097992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.098156 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.098238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.098406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.098492 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.098645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.098792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.098919 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.099197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.099279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.099382 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] publish_errors = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.099600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.099676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.099831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.099921 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.100001 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.100098 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.100238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.100382 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.100459 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.100690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] standard_threads = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.100870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.100955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.101057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] publish_errors = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.101151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.101295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.101372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.101466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] use_journal = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.101540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.101659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] use_json = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.101733 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.101807 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] use_stderr = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.101935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.102017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] use_syslog = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.102174 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.102261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.102413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] standard_threads = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.102490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.102649 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.102760 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.102926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.103010 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.103105 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.103182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.103284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] use_journal = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.103484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.103562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] use_json = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.103644 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] use_stderr = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.103731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.103802 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] use_syslog = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.103870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.104013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:58:32.104110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.104270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.104348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.104519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.104596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.104785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.104862 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.105029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.105107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.105275 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.105350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.105514 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.105596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.105760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.105843 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.105998 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.106078 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.106230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.106304 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.106483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.106566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.106754 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.106832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.106995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.107081 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.107273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.107357 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.107466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.107542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.107721 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.107850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.107958 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.108038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.108202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.108285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.108427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.108511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.108656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.108735 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.108884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.108961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.109118 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.109196 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.109366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.109445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.109612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.109689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.109841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.109919 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.110096 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.110265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.110350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.110577 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.110654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.110848 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.110930 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.111099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.111178 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.111329 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.111406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.111564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.111643 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.111799 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.111876 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.112033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.112116 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.112264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.112342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.112492 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.112569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.112720 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.112797 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.112967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.113045 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.113188 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.113266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.113418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.113495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.113654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.113739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.113883 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.113960 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.114115 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.114192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.114348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.114426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.114587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.114664 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.114848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.114929 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.115084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.115162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.115322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.115409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.115559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.115637 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.115792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.115871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.116142 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.116267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.116267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.116438 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.116512 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.116573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.116698 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.116780 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.116927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.117047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.117166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.117259 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.117401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.117487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.117632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.117710 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.117864 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.117958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.118099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.118193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.118332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.118409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.118588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.118702 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.118857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.118974 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.119083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.119176 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.119324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.119421 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.119548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.119633 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.119775 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.119861 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.120002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.120130 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.120261 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.120396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.120495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.120615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.120692 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.120779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.120943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.121022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.121183 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.121270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.121421 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.121507 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.121657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.121743 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.121892 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.121969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.122128 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.122212 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.122385 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.122463 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.122622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.122730 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.122886 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.122973 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.123123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.123201 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.123363 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.123440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.123600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.123693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.123836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.123914 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.124070 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.124147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.124301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.124458 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.124533 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.124603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.124747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.124824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.124978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.125067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.125206 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.125296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.125450 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.125531 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.125708 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.125784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.125957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.126047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.126281 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.126367 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.126441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.126517 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.126692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.126785 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.126929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.127007 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.127157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] auth.external = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.127240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.127386 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.127471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.127627 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.127704 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.127868 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.127946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.128112 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] auth.password = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.128195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.128341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] auth.token = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.128419 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.128577 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.128654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.128811 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.128903 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.129056 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.129144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.129286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.129364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] auth.external = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.129519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.129595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.129762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.129841 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.129992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.130069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.130226 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] credential.caching = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.130304 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] auth.password = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.130470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.130549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] auth.token = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.130732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.130824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.130975 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.131054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.131209 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.131287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.131449 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.131528 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.131679 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.131756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.131936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.132015 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.132172 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.132259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.132411 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] credential.caching = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.132490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.132661 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.132740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.132915 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.132993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.133167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.133245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.133409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.133488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.133656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.133735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.133895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.133981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.134128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] federation.caching = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.134206 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.134373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.134452 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.134621 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.134735 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.134895 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.134975 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.135126 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.135204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.135377 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.135461 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.135613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.135693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.135866 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.135950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.136102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.136180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.136355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.136434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] federation.caching = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.136592 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.136670 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.136839 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.136916 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.137075 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity.caching = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.137160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.137322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.137400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.137568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.137645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.137803 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.137882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.138036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.138114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.138286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.138434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.138547 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.138631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.138814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.138894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.139068 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.139158 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.139308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity.caching = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.139387 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.139561 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.139638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.139791 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.139883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.140022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.140101 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.140265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.140342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.140509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.140627 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.140756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.140867 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.141012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.141115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.141794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.142198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.142281 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.142353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.142424 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.142502 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.142575 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.142714 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.142798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.142960 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.143039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.143207 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.143283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.143443 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.143524 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.143682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.143760 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.143922 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.144001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.144170 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.144250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.144403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.144487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.144650 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.144728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.144896 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.144977 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.145132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.145210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.145375 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.145453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.145613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.145693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.145841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.145918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.146079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.146157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.146310 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.146390 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.146548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.146631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.146806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.146878 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.147050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.147129 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.147288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.147366 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.147527 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.147605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.147778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.147857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.148022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.148099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.148254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.148332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.148483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.148562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.148718 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.148796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.148962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.149040 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.149197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.149277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.149434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.149539 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.149668 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.149748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.149901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.149990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.150138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.150217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.150393 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.150478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.150622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.150725 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.150882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.150960 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.151128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.151218 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.151380 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.151458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.151612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.151689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.151842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.151928 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.152075 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.152154 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.152308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.152385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.152540 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.152625 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.152770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.152849 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.153015 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.153094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.153262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.153352 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.153507 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.153586 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.153756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.153835 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.153996 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.154081 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.154222 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.154300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.154465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.154543 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.154719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.154813 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.154960 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.155039 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.155192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.155269 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.155425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.155510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.155669 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.155748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.155917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.155995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.156153 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.156239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.156405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.156484 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.156636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.156715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.156897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.157016 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.157726 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.157808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.157966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.158090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.158198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.158288 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.158446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.158525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.158695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.158777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.158939 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.159030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.159173 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.159253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.159411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] resource.caching = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.159490 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.159659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.159749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.159905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.159984 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.160141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.160223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.160389 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.160476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.160636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.160713 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.160867 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.160945 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.161111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.161199 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.161344 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.161423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.161579 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.161657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.161813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] role.caching = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.161900 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] resource.caching = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.162058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] role.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.162137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.162287 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.162366 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.162532 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.162712 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.162790 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.162862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.163038 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.163117 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.163282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.163373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.163525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.163603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.163766 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.163844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.164007 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.164094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.164244 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] role.caching = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.164322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.164476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.164554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] role.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.164724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.164803 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.164967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.165045 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.165218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.165297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.165464 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.165550 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.165716 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.165860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.165967 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.166100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.166219 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.166352 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.166469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.166603 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.166752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.166878 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.166997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.167131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.167253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.167368 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.167507 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.167622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.167761 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.167873 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.168013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.168113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.168346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.168424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.168594 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.168674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.168815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.168920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.169064 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.169145 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.169318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.169399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.169566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.169646 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.169802 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.169880 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.170037 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.170117 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.170268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.170346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.170502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] token.caching = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.170588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.170762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.170845 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.171012 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.171089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.171245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.171334 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.171499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.171578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.171744 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.171823 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.171977 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.172063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.172219 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.172297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.172457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.172706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.172782 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.172853 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.172972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] token.caching = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.173102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.173220 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.173329 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.173473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.173563 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.173713 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.173794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.173973 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.174053 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.174226 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.174313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.174471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.174550 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.174738 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.174820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.174989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.175081 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.connection = **** {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.175241 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.175334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.175481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.175561 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.175724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.175807 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.175962 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.176050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.176198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.176276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.176436 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.176515 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.176672 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.176759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.176910 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.176990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.177154 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.177233 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.177398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.177511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.connection = **** {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.177640 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.177737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.177877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.177958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.178126 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.178206 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.178362 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.178443 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.178607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.178742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.178864 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.178945 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.179106 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.179187 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.179338 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.179416 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.179507 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80333) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 08:58:32.179586 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.179814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.180072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.180315 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.180571 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.180814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.181064 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.181301 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.181546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.181786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:58:32.181959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80334) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 08:58:32.211229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.211455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.211680 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.212721 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.212958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.213184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.220490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.222267 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.248655 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.248883 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.250863 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.251132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.390506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.390780 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.440168 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.440396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.441286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.441528 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.505152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.505334 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.505451 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.505530 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.505631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.505711 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.505785 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.505868 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.505988 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.506072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.506205 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.506289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.531161 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.531307 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.531391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.531487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.531602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.531687 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.531827 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.532480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.532587 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.532795 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.532878 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.533012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.533183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.533407 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.544143 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.544483 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.557526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.557778 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.557982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.558132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.558263 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.558378 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.558466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.558600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.558703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.558840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80333) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.558923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.559061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.559280 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.559500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80334) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:58:32.577256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.577413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.577501 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 08:58:32.577622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 08:58:32.577792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.577955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.578098 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 08:58:32.578178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 08:58:32.578240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.578404 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.578496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.578660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.579435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.579521 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.579596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.579736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.590802 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.591081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.591352 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.591637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.594307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.595178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.595579 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.595860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.600415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.603745 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.603954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.604038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.604289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.604683 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.604969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.605231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.605504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.605781 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.606053 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.606325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.606597 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.607231 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.607509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.607783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.608168 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.608446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.608701 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.608965 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.609233 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.609497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.609763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.610052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.633419 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.635071 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.635355 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.636741 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.638350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.638626 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.640682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.640923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.641206 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.644437 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.644709 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.644984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.649082 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.649369 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.652233 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.652511 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.654070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.654358 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.654640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.654946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.655225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.655502 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.655773 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.656041 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.657783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.658227 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.658505 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.658808 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.659081 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.659350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.659615 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.659877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.675362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.675640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.678535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.678831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.680387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.683441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.684058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.684354 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.687056 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.687347 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.689457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.693001 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.693134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.696868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.697140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.697440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.697734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.698029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.698311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.698590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.700633 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.700924 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.701212 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.701499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.701772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.702051 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.718175 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.718552 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.718850 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.719121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.722058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.722428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.722719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.722992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.727680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.729389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.729668 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.730877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.732556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.732825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.732953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.733157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.733437 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.733717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.734004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.735933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.736209 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.736483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.736756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.737025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.747605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.747886 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.748168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.748459 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.748739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.749029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.749302 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.750389 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.750734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.751006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.751287 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.751561 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.751845 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.752113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.769341 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.771765 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.772657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.775043 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.775988 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.777610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.778347 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.779277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.779559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.779847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.779974 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.781527 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.781803 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.782635 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.788036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.789662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.790169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.791792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.792913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.793205 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.793491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.793772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.795012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.795297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.795574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.795849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.803614 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.803903 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.804185 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.805616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.805894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.806166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.812105 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80334) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.812386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.812669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.812947 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.813803 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.814088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.814533 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.814660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80333) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:58:32.814829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.814957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.815083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.815208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.815360 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.815498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.815624 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.815768 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.815904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.816081 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.816209 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.816334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.816459 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80334) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.816584 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.816717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.816950 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.817225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.817481 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.817747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.818006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80333) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:58:32.855376 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555ab68dc9c0 pid: 80334 (default app) Jul 20 08:58:32.857113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555ab68dc9c0 pid: 80333 (default app) Jul 20 08:58:32.858727 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cfcc9-7152-402a-8b82-674c03153b36 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:32.859030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cfcc9-7152-402a-8b82-674c03153b36 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:32.859318 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cfcc9-7152-402a-8b82-674c03153b36 None None] PATH_INFO: `/v3` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:32.864087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1/1] 10.222.0.44 () {50 vars in 791 bytes} [Wed Jul 20 08:58:32 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:58:40.480769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2618af3-71e9-4014-b0b4-38ba169a0eef None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:40.481348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2618af3-71e9-4014-b0b4-38ba169a0eef None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:40.481348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2618af3-71e9-4014-b0b4-38ba169a0eef None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:40.486796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1/2] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:58:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:58:40.492002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad23a0-e8a5-42fe-9b54-b5cee18a083c None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:40.492388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad23a0-e8a5-42fe-9b54-b5cee18a083c None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:40.492556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad23a0-e8a5-42fe-9b54-b5cee18a083c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:40.561312 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: INFO dbcounter [None req-46ad23a0-e8a5-42fe-9b54-b5cee18a083c None None] Registered counter for database keystone Jul 20 08:58:40.583188 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG oslo_db.sqlalchemy.engines [None req-46ad23a0-e8a5-42fe-9b54-b5cee18a083c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80334) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 08:58:40.589399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writer thread running {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 08:58:40.762793 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG passlib.handlers.bcrypt [None req-46ad23a0-e8a5-42fe-9b54-b5cee18a083c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80334) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 08:58:40.763124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG passlib.handlers.bcrypt [None req-46ad23a0-e8a5-42fe-9b54-b5cee18a083c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80334) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 08:58:40.817518 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-46ad23a0-e8a5-42fe-9b54-b5cee18a083c None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:58:40.819392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-46ad23a0-e8a5-42fe-9b54-b5cee18a083c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:58:40.941034 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 2/3] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:58:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 08:58:41.139091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2618af3-71e9-4014-b0b4-38ba169a0eef None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:41.208522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: INFO dbcounter [None req-d2618af3-71e9-4014-b0b4-38ba169a0eef None None] Registered counter for database keystone Jul 20 08:58:41.228226 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG oslo_db.sqlalchemy.engines [None req-d2618af3-71e9-4014-b0b4-38ba169a0eef None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80333) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 08:58:41.234283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writer thread running {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 08:58:41.355228 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2618af3-71e9-4014-b0b4-38ba169a0eef None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:41.359514 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ece585-d2fa-4e63-bc8c-4760108a7e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:41.361273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ece585-d2fa-4e63-bc8c-4760108a7e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:41.361564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ece585-d2fa-4e63-bc8c-4760108a7e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:41.361845 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ece585-d2fa-4e63-bc8c-4760108a7e59 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:41.369906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ece585-d2fa-4e63-bc8c-4760108a7e59 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:41.422057 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d7ece585-d2fa-4e63-bc8c-4760108a7e59 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 08:58:41.423401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ece585-d2fa-4e63-bc8c-4760108a7e59 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:41.432972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 2/4] 10.222.0.44 () {58 vars in 1324 bytes} [Wed Jul 20 08:58:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 08:58:41.438955 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad23a0-e8a5-42fe-9b54-b5cee18a083c None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:41.453246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad23a0-e8a5-42fe-9b54-b5cee18a083c None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:41.457874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf1afb-2b47-4bce-ad2c-ff3edd5b29c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:41.459028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf1afb-2b47-4bce-ad2c-ff3edd5b29c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:41.459315 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf1afb-2b47-4bce-ad2c-ff3edd5b29c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:41.459594 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf1afb-2b47-4bce-ad2c-ff3edd5b29c7 admin admin] PATH_INFO: `/v3/projects/d5e32bdaa14a4d31baa44b436b02a242` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:41.461061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf1afb-2b47-4bce-ad2c-ff3edd5b29c7 admin admin] RBAC: Authorizing `identity:get_project(project_id=d5e32bdaa14a4d31baa44b436b02a242)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:41.515138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b8bf1afb-2b47-4bce-ad2c-ff3edd5b29c7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 08:58:41.515670 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf1afb-2b47-4bce-ad2c-ff3edd5b29c7 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:41.520600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 3/5] 10.222.0.44 () {56 vars in 1207 bytes} [Wed Jul 20 08:58:41 2022] GET /identity/v3/projects/d5e32bdaa14a4d31baa44b436b02a242 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:58:43.512961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc932a9-3ff8-4503-9ed7-8f5f9c03ffc5 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:43.513506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc932a9-3ff8-4503-9ed7-8f5f9c03ffc5 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:43.513506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc932a9-3ff8-4503-9ed7-8f5f9c03ffc5 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:43.517902 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 3/6] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:58:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:58:43.523373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b169bb-5539-4586-b673-942ead680f76 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:43.523804 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b169bb-5539-4586-b673-942ead680f76 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:43.523932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b169bb-5539-4586-b673-942ead680f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:43.543103 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-33b169bb-5539-4586-b673-942ead680f76 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:58:43.544617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-33b169bb-5539-4586-b673-942ead680f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:58:43.553272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 4/7] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:58:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 08:58:43.575120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc932a9-3ff8-4503-9ed7-8f5f9c03ffc5 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:43.587059 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc932a9-3ff8-4503-9ed7-8f5f9c03ffc5 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:43.591235 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4bd039-83c7-4a6f-b456-bdc5a14f02c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:43.591864 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bd039-83c7-4a6f-b456-bdc5a14f02c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:43.592147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bd039-83c7-4a6f-b456-bdc5a14f02c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:43.592426 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bd039-83c7-4a6f-b456-bdc5a14f02c6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:43.600525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4bd039-83c7-4a6f-b456-bdc5a14f02c6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:43.602272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4bd039-83c7-4a6f-b456-bdc5a14f02c6 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:43.611518 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 4/8] 10.222.0.44 () {58 vars in 1324 bytes} [Wed Jul 20 08:58:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 08:58:43.616861 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b169bb-5539-4586-b673-942ead680f76 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:43.626123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b169bb-5539-4586-b673-942ead680f76 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:43.630540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b25ae-9b2a-4938-bd66-0af9bc35789e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:43.631440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b25ae-9b2a-4938-bd66-0af9bc35789e admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:43.631726 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b25ae-9b2a-4938-bd66-0af9bc35789e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:43.632006 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b25ae-9b2a-4938-bd66-0af9bc35789e admin admin] PATH_INFO: `/v3/users/d01511dd98c7416fb8387a8557165556` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:43.633391 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b25ae-9b2a-4938-bd66-0af9bc35789e admin admin] RBAC: Authorizing `identity:get_user(user_id=d01511dd98c7416fb8387a8557165556)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:43.635251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b25ae-9b2a-4938-bd66-0af9bc35789e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:43.636852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 5/9] 10.222.0.44 () {56 vars in 1198 bytes} [Wed Jul 20 08:58:43 2022] GET /identity/v3/users/d01511dd98c7416fb8387a8557165556 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:58:45.706335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f966a0-def7-410e-a50a-3bd395205f21 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:45.706902 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f966a0-def7-410e-a50a-3bd395205f21 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:45.706902 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f966a0-def7-410e-a50a-3bd395205f21 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:45.708287 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 5/10] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:58:45.713485 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09719e3e-861b-4b51-b0d4-f7d8499fd913 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:45.713913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09719e3e-861b-4b51-b0d4-f7d8499fd913 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:45.714058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09719e3e-861b-4b51-b0d4-f7d8499fd913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:45.731215 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-09719e3e-861b-4b51-b0d4-f7d8499fd913 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:58:45.732398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-09719e3e-861b-4b51-b0d4-f7d8499fd913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:58:45.760898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc5c05-d465-4b55-8054-df9f5d10ffce None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:45.761191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc5c05-d465-4b55-8054-df9f5d10ffce None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:45.761474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc5c05-d465-4b55-8054-df9f5d10ffce None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:45.762870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 6/11] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:58:45.768018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f25f2-a517-4377-be0d-916e5aa035bb None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:45.768308 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f25f2-a517-4377-be0d-916e5aa035bb None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:45.768644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f25f2-a517-4377-be0d-916e5aa035bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:45.808805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG passlib.handlers.bcrypt [None req-0b1f25f2-a517-4377-be0d-916e5aa035bb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80333) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 08:58:45.809152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG passlib.handlers.bcrypt [None req-0b1f25f2-a517-4377-be0d-916e5aa035bb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80333) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 08:58:45.814165 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 6/12] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:58:45.837031 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09719e3e-861b-4b51-b0d4-f7d8499fd913 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:45.855005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-0b1f25f2-a517-4377-be0d-916e5aa035bb None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:58:45.856589 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-0b1f25f2-a517-4377-be0d-916e5aa035bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:58:45.861465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:45.942270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09719e3e-861b-4b51-b0d4-f7d8499fd913 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:45.960573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 7/13] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:58:45.976502 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582adfe9-4dad-478f-94a2-16c530b89cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:45.977296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582adfe9-4dad-478f-94a2-16c530b89cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:45.977576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582adfe9-4dad-478f-94a2-16c530b89cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:45.977851 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582adfe9-4dad-478f-94a2-16c530b89cd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:45.983995 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f25f2-a517-4377-be0d-916e5aa035bb None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:45.989715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582adfe9-4dad-478f-94a2-16c530b89cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:58:46.000854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-582adfe9-4dad-478f-94a2-16c530b89cd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:58:46.000854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:58:46.000854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:58:46.000854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:58:46.000854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:58:46.000854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:58:46.000854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:58:46.000854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:58:46.000854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:58:46.000854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:58:46.000854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:58:46.000854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:58:46.000854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:58:46.000854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:58:46.000854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:58:46.000854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:58:46.001937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:58:46.001937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:58:46.001937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:58:46.001937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:58:46.001937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:58:46.001937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:58:46.001937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:58:46.001937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:58:46.001937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:58:46.001937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:58:46.001937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:58:46.001937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:58:46.001937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:58:46.001937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:58:46.001937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:58:46.001937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:58:46.002829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:58:46.002829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:58:46.002829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:58:46.002829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:58:46.002829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:58:46.002829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:58:46.002829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:58:46.002829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:58:46.002829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:58:46.002829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:58:46.002829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:58:46.002829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:58:46.002829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 7/14] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 08:58:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:58:46.007795 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582adfe9-4dad-478f-94a2-16c530b89cd2 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:46.088117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f25f2-a517-4377-be0d-916e5aa035bb None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:46.089513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:46.091124 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:46.108567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582adfe9-4dad-478f-94a2-16c530b89cd2 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:46.126066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f9a5e-0f56-448e-bd70-161b853ab684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:46.127004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f9a5e-0f56-448e-bd70-161b853ab684 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:46.127295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f9a5e-0f56-448e-bd70-161b853ab684 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:46.127575 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f9a5e-0f56-448e-bd70-161b853ab684 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:46.143140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecf83a-ca93-40d8-b93f-9ea4633e12a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:46.143808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecf83a-ca93-40d8-b93f-9ea4633e12a7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:46.144096 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecf83a-ca93-40d8-b93f-9ea4633e12a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:46.144384 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecf83a-ca93-40d8-b93f-9ea4633e12a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:46.145140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ecf83a-ca93-40d8-b93f-9ea4633e12a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:58:46.145448 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ecf83a-ca93-40d8-b93f-9ea4633e12a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:46.146890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ecf83a-ca93-40d8-b93f-9ea4633e12a7 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:46.159087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 8/15] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:58:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:58:46.164562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecf83a-ca93-40d8-b93f-9ea4633e12a7 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:46.196466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f9a5e-0f56-448e-bd70-161b853ab684 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:46.197997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f9a5e-0f56-448e-bd70-161b853ab684 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:46.265445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecf83a-ca93-40d8-b93f-9ea4633e12a7 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:46.299047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 8/16] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:58:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 08:58:46.300542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e974278-31d9-405d-a111-439cb9bdd616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:46.301414 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e974278-31d9-405d-a111-439cb9bdd616 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:46.301696 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e974278-31d9-405d-a111-439cb9bdd616 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:46.301971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e974278-31d9-405d-a111-439cb9bdd616 None admin] PATH_INFO: `/v3/users/d01511dd98c7416fb8387a8557165556` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:46.303217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e974278-31d9-405d-a111-439cb9bdd616 None admin] RBAC: Authorizing `identity:get_user(user_id=d01511dd98c7416fb8387a8557165556)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:46.304680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e974278-31d9-405d-a111-439cb9bdd616 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:46.305006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f9a5e-0f56-448e-bd70-161b853ab684 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:46.306272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 9/17] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:58:46 2022] GET /identity/v3/users/d01511dd98c7416fb8387a8557165556 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:58:46.308710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:46.311599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e974278-31d9-405d-a111-439cb9bdd616 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:46.326805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:46.405000 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f9a5e-0f56-448e-bd70-161b853ab684 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:46.412101 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e974278-31d9-405d-a111-439cb9bdd616 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:46.439729 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68d049-d708-4807-ab18-acdca2a27dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:46.440520 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68d049-d708-4807-ab18-acdca2a27dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:46.440807 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68d049-d708-4807-ab18-acdca2a27dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:46.441084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68d049-d708-4807-ab18-acdca2a27dc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:46.445645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18125d4f-0c8c-47ca-aeba-6e7d9cffe46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:46.446433 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18125d4f-0c8c-47ca-aeba-6e7d9cffe46b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:46.446746 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18125d4f-0c8c-47ca-aeba-6e7d9cffe46b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:46.447033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18125d4f-0c8c-47ca-aeba-6e7d9cffe46b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:46.448223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18125d4f-0c8c-47ca-aeba-6e7d9cffe46b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:46.449679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18125d4f-0c8c-47ca-aeba-6e7d9cffe46b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:46.451316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 10/18] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:58:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:58:46.457530 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18125d4f-0c8c-47ca-aeba-6e7d9cffe46b None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:46.460136 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68d049-d708-4807-ab18-acdca2a27dc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:46.461596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68d049-d708-4807-ab18-acdca2a27dc0 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:46.484260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-1c68d049-d708-4807-ab18-acdca2a27dc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:58:46.484260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:58:46.484260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:58:46.484260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:58:46.484260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:58:46.484260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:58:46.484260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:58:46.484260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:58:46.484260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:58:46.484260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:58:46.484260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:58:46.484260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:58:46.484260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:58:46.484260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:58:46.484260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:58:46.485140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:58:46.485930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:58:46.485930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:58:46.485930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:58:46.485930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:58:46.485930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 9/19] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:58:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:58:46.491227 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68d049-d708-4807-ab18-acdca2a27dc0 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:46.540619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:46.549717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:46.562479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18125d4f-0c8c-47ca-aeba-6e7d9cffe46b None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:46.596026 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68d049-d708-4807-ab18-acdca2a27dc0 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:46.599645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33856caa-47eb-4666-8884-7a2d3e1759e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:46.600309 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33856caa-47eb-4666-8884-7a2d3e1759e9 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:46.600599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33856caa-47eb-4666-8884-7a2d3e1759e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:46.600883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33856caa-47eb-4666-8884-7a2d3e1759e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:46.601723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33856caa-47eb-4666-8884-7a2d3e1759e9 None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.domain.id=default, user.id=d01511dd98c7416fb8387a8557165556) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:58:46.602042 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33856caa-47eb-4666-8884-7a2d3e1759e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:46.603546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33856caa-47eb-4666-8884-7a2d3e1759e9 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:46.610897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 11/20] 10.222.0.44 () {56 vars in 1322 bytes} [Wed Jul 20 08:58:46 2022] GET /identity/v3/role_assignments?user.id=d01511dd98c7416fb8387a8557165556&scope.domain.id=default&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:58:46.630781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b20d3-d5b2-483b-91ed-e50c6b243b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:46.631474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b20d3-d5b2-483b-91ed-e50c6b243b98 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:46.631791 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b20d3-d5b2-483b-91ed-e50c6b243b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:46.632073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b20d3-d5b2-483b-91ed-e50c6b243b98 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:46.632836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b20d3-d5b2-483b-91ed-e50c6b243b98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:58:46.633147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b20d3-d5b2-483b-91ed-e50c6b243b98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:46.634595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b20d3-d5b2-483b-91ed-e50c6b243b98 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:46.654138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 10/21] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 08:58:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:58:48.837498 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61f64d-5ee7-4930-851e-f52581a4914b None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:48.838335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61f64d-5ee7-4930-851e-f52581a4914b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:48.838787 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61f64d-5ee7-4930-851e-f52581a4914b None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:48.840334 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 12/22] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:58:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:58:48.846088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e14f5-7bca-4c33-87b1-1a9740e76c16 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:48.846901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e14f5-7bca-4c33-87b1-1a9740e76c16 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:48.847328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e14f5-7bca-4c33-87b1-1a9740e76c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:48.869281 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-2a0e14f5-7bca-4c33-87b1-1a9740e76c16 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:58:48.870781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2a0e14f5-7bca-4c33-87b1-1a9740e76c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:58:48.959283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207efce-9992-407e-81f9-0c010df776d6 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:48.959921 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207efce-9992-407e-81f9-0c010df776d6 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:48.960420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207efce-9992-407e-81f9-0c010df776d6 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:48.961991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 13/23] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:58:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:58:48.962856 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 11/24] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:58:48.967936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b716b1-603f-44ce-83f0-50539f239259 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:48.968231 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b716b1-603f-44ce-83f0-50539f239259 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:48.968515 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b716b1-603f-44ce-83f0-50539f239259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:48.985664 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-04b716b1-603f-44ce-83f0-50539f239259 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:58:48.990911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-04b716b1-603f-44ce-83f0-50539f239259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:58:49.005632 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207efce-9992-407e-81f9-0c010df776d6 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:49.045182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:49.082128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 12/25] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:58:49.122701 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b716b1-603f-44ce-83f0-50539f239259 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:49.200797 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207efce-9992-407e-81f9-0c010df776d6 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:49.260788 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf9694-e5cc-4625-ba67-a779593659d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:49.261946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf9694-e5cc-4625-ba67-a779593659d7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:49.262427 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf9694-e5cc-4625-ba67-a779593659d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:49.262947 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf9694-e5cc-4625-ba67-a779593659d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:49.284117 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf9694-e5cc-4625-ba67-a779593659d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:58:49.296263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-aeaf9694-e5cc-4625-ba67-a779593659d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:58:49.296263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:58:49.296263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:58:49.296263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:58:49.296263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:58:49.296263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:58:49.296263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:58:49.296263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:58:49.296263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:58:49.296263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:58:49.296263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:58:49.296263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:58:49.296263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:58:49.296263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:58:49.296263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:58:49.296263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:58:49.298073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:58:49.298073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:58:49.298073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:58:49.298073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:58:49.298073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:58:49.298073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:58:49.298073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:58:49.298073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:58:49.298073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:58:49.298073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:58:49.298073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:58:49.298073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:58:49.298073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:58:49.298073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:58:49.298073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:58:49.298073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:58:49.299038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:58:49.299038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:58:49.299038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:58:49.299038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:58:49.299038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:58:49.299038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:58:49.299038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:58:49.299038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:58:49.299038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:58:49.299038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:58:49.299038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:58:49.299038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:58:49.301791 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 14/26] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 08:58:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:58:49.307427 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf9694-e5cc-4625-ba67-a779593659d7 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:49.311991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:49.372634 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b716b1-603f-44ce-83f0-50539f239259 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:49.443986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2f560-5305-49a0-80a6-50bc1ee78a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:49.444577 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2f560-5305-49a0-80a6-50bc1ee78a30 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:49.444865 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2f560-5305-49a0-80a6-50bc1ee78a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:49.445146 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2f560-5305-49a0-80a6-50bc1ee78a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:49.457535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf9694-e5cc-4625-ba67-a779593659d7 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:49.463854 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2f560-5305-49a0-80a6-50bc1ee78a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:49.465484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2f560-5305-49a0-80a6-50bc1ee78a30 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:49.496763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-1bb2f560-5305-49a0-80a6-50bc1ee78a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:58:49.496763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:58:49.496763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:58:49.496763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:58:49.496763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:58:49.496763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:58:49.496763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:58:49.496763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:58:49.496763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:58:49.496763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:58:49.496763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:58:49.496763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:58:49.496763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:58:49.496763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:58:49.496763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:58:49.497645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:58:49.498460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:58:49.498460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:58:49.498460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:58:49.498460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:58:49.498460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 13/27] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:58:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:58:49.498460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3c4d3-da65-4187-8c1b-5784bf6049fb None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:49.498460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3c4d3-da65-4187-8c1b-5784bf6049fb None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:49.498460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3c4d3-da65-4187-8c1b-5784bf6049fb None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:49.500419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 14/28] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:58:49.502562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3c4d3-da65-4187-8c1b-5784bf6049fb None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:49.511979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2a10a-3fd9-4c90-b292-63d5dd1a654a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:49.512669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2a10a-3fd9-4c90-b292-63d5dd1a654a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:49.512954 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2a10a-3fd9-4c90-b292-63d5dd1a654a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:49.513235 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2a10a-3fd9-4c90-b292-63d5dd1a654a None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:49.514001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2a10a-3fd9-4c90-b292-63d5dd1a654a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:58:49.514311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2a10a-3fd9-4c90-b292-63d5dd1a654a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:49.517162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2a10a-3fd9-4c90-b292-63d5dd1a654a None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:49.528315 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 15/29] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:58:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:58:49.530298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c546f-690d-40a8-beee-e64d2331efc3 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:49.530593 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c546f-690d-40a8-beee-e64d2331efc3 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:49.530921 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c546f-690d-40a8-beee-e64d2331efc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:49.550569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-ec9c546f-690d-40a8-beee-e64d2331efc3 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:58:49.552050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ec9c546f-690d-40a8-beee-e64d2331efc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:58:49.580418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:49.652282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:49.668828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 16/30] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:58:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:58:49.669780 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c546f-690d-40a8-beee-e64d2331efc3 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:49.748050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3c4d3-da65-4187-8c1b-5784bf6049fb None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:49.819264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d724a-4603-427f-b43b-43e0d5cac3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:49.819943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d724a-4603-427f-b43b-43e0d5cac3ca None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:49.820229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d724a-4603-427f-b43b-43e0d5cac3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:49.820507 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d724a-4603-427f-b43b-43e0d5cac3ca None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:49.821245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d724a-4603-427f-b43b-43e0d5cac3ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:58:49.821555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d724a-4603-427f-b43b-43e0d5cac3ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:49.822982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d724a-4603-427f-b43b-43e0d5cac3ca None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:49.834979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c546f-690d-40a8-beee-e64d2331efc3 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:49.842091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 15/31] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 08:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:58:49.843113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d724a-4603-427f-b43b-43e0d5cac3ca None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:49.874916 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a60374-c69f-40d0-8239-77add54fa0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:49.875907 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a60374-c69f-40d0-8239-77add54fa0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:49.876238 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a60374-c69f-40d0-8239-77add54fa0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:49.876560 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a60374-c69f-40d0-8239-77add54fa0c6 None admin] PATH_INFO: `/v3/users/d01511dd98c7416fb8387a8557165556` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:49.878019 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a60374-c69f-40d0-8239-77add54fa0c6 None admin] RBAC: Authorizing `identity:get_user(user_id=d01511dd98c7416fb8387a8557165556)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:49.879658 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a60374-c69f-40d0-8239-77add54fa0c6 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:49.881713 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 17/32] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:58:49 2022] GET /identity/v3/users/d01511dd98c7416fb8387a8557165556 => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:58:49.887820 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bafb14-9208-4625-8b47-6485cd7d18d3 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:49.888277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bafb14-9208-4625-8b47-6485cd7d18d3 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:49.888740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bafb14-9208-4625-8b47-6485cd7d18d3 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:49.890603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 18/33] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:58:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:58:49.892066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bafb14-9208-4625-8b47-6485cd7d18d3 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:49.938657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:49.952740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d724a-4603-427f-b43b-43e0d5cac3ca None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:49.967933 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:49.992699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fa01a-c911-42f1-9341-cfe0ca0fd73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:49.993359 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fa01a-c911-42f1-9341-cfe0ca0fd73a None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:49.993646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fa01a-c911-42f1-9341-cfe0ca0fd73a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:49.993952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fa01a-c911-42f1-9341-cfe0ca0fd73a None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:49.994829 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9fa01a-c911-42f1-9341-cfe0ca0fd73a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:49.996216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9fa01a-c911-42f1-9341-cfe0ca0fd73a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:50.011757 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bafb14-9208-4625-8b47-6485cd7d18d3 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:50.048995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31c856-676e-479b-9cff-403c0fc75ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:50.049889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31c856-676e-479b-9cff-403c0fc75ee9 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:50.050339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31c856-676e-479b-9cff-403c0fc75ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:50.050781 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31c856-676e-479b-9cff-403c0fc75ee9 None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:50.051643 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31c856-676e-479b-9cff-403c0fc75ee9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:50.053359 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31c856-676e-479b-9cff-403c0fc75ee9 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:50.124596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 16/34] 10.222.0.44 () {60 vars in 1131 bytes} [Wed Jul 20 08:58:49 2022] POST /identity/v3/roles => generated 219 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:58:50.125769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fa01a-c911-42f1-9341-cfe0ca0fd73a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:50.201021 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 19/35] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jul 20 08:58:49 2022] POST /identity/v3/projects => generated 301 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:58:50.202952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a68a5d-f582-4025-b5d6-67f2402c391d None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:50.204043 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a68a5d-f582-4025-b5d6-67f2402c391d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:50.204857 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a68a5d-f582-4025-b5d6-67f2402c391d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:50.237556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a7a68a5d-f582-4025-b5d6-67f2402c391d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:58:50.239618 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a7a68a5d-f582-4025-b5d6-67f2402c391d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:58:50.280435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fa01a-c911-42f1-9341-cfe0ca0fd73a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:50.353943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636f8e5-c1f2-4c84-bc33-5b323b8ca2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:50.355055 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636f8e5-c1f2-4c84-bc33-5b323b8ca2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:50.355593 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636f8e5-c1f2-4c84-bc33-5b323b8ca2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:50.356081 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636f8e5-c1f2-4c84-bc33-5b323b8ca2d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:50.361194 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636f8e5-c1f2-4c84-bc33-5b323b8ca2d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:50.363219 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636f8e5-c1f2-4c84-bc33-5b323b8ca2d9 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:50.365640 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 17/36] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:58:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:58:50.372769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636f8e5-c1f2-4c84-bc33-5b323b8ca2d9 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:50.396428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:50.424645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 20/37] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:58:50.479773 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a68a5d-f582-4025-b5d6-67f2402c391d None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:50.506090 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636f8e5-c1f2-4c84-bc33-5b323b8ca2d9 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:50.551428 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:50.561791 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24477398-38ea-4032-b580-c5471d69bae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:50.565119 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24477398-38ea-4032-b580-c5471d69bae0 None admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:50.565119 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24477398-38ea-4032-b580-c5471d69bae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:50.565119 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24477398-38ea-4032-b580-c5471d69bae0 None admin] PATH_INFO: `/v3/domains/default/users/d01511dd98c7416fb8387a8557165556/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:50.568554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24477398-38ea-4032-b580-c5471d69bae0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d01511dd98c7416fb8387a8557165556, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:50.571435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24477398-38ea-4032-b580-c5471d69bae0 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:50.598302 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 18/38] 10.222.0.44 () {58 vars in 1361 bytes} [Wed Jul 20 08:58:50 2022] PUT /identity/v3/domains/default/users/d01511dd98c7416fb8387a8557165556/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:58:50.600229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adf3df-d0b6-408e-9cd7-f685ca1242e2 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:50.600742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adf3df-d0b6-408e-9cd7-f685ca1242e2 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:50.601244 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adf3df-d0b6-408e-9cd7-f685ca1242e2 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:50.603722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 19/39] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:58:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:58:50.612617 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb79263-6fe3-42de-8ccf-3b8089d529aa None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:50.612934 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb79263-6fe3-42de-8ccf-3b8089d529aa None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:50.613195 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb79263-6fe3-42de-8ccf-3b8089d529aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:50.633414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-eeb79263-6fe3-42de-8ccf-3b8089d529aa None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:58:50.635457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-eeb79263-6fe3-42de-8ccf-3b8089d529aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:58:50.645809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a68a5d-f582-4025-b5d6-67f2402c391d None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:50.689213 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9d690-b907-409a-bc45-c0f3913a4b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:50.689904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9d690-b907-409a-bc45-c0f3913a4b93 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:50.690191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9d690-b907-409a-bc45-c0f3913a4b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:50.690473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9d690-b907-409a-bc45-c0f3913a4b93 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:50.691276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec9d690-b907-409a-bc45-c0f3913a4b93 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:50.692861 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec9d690-b907-409a-bc45-c0f3913a4b93 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:50.712980 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 21/40] 10.222.0.44 () {60 vars in 1131 bytes} [Wed Jul 20 08:58:50 2022] POST /identity/v3/roles => generated 225 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:58:50.720439 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 20/41] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:58:50.743906 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9d690-b907-409a-bc45-c0f3913a4b93 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:50.768345 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4f54b-32b3-455b-af79-7614c9e56f08 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:50.769006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4f54b-32b3-455b-af79-7614c9e56f08 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:50.769536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4f54b-32b3-455b-af79-7614c9e56f08 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:50.771469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 21/42] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:58:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:58:50.777772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8136ac-5ce8-4969-9f6e-155a61996530 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:50.778121 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8136ac-5ce8-4969-9f6e-155a61996530 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:50.778360 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8136ac-5ce8-4969-9f6e-155a61996530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:50.792139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:50.805965 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-7b8136ac-5ce8-4969-9f6e-155a61996530 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:58:50.807513 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7b8136ac-5ce8-4969-9f6e-155a61996530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:58:50.899242 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 22/43] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:58:50.924666 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8136ac-5ce8-4969-9f6e-155a61996530 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:50.992037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9d690-b907-409a-bc45-c0f3913a4b93 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:51.000193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:51.032385 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934e4c5-61e1-4de7-9d68-82d151b94193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:51.033070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934e4c5-61e1-4de7-9d68-82d151b94193 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:51.033355 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934e4c5-61e1-4de7-9d68-82d151b94193 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:51.033640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934e4c5-61e1-4de7-9d68-82d151b94193 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:51.034392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e934e4c5-61e1-4de7-9d68-82d151b94193 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:51.035923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e934e4c5-61e1-4de7-9d68-82d151b94193 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:51.044530 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8136ac-5ce8-4969-9f6e-155a61996530 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:51.049046 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 22/44] 10.222.0.44 () {60 vars in 1131 bytes} [Wed Jul 20 08:58:50 2022] POST /identity/v3/roles => generated 223 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:58:51.085322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f27db2-5e6b-4ffe-b2b3-d715eed58293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:51.086067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f27db2-5e6b-4ffe-b2b3-d715eed58293 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:51.086369 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f27db2-5e6b-4ffe-b2b3-d715eed58293 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:51.086637 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f27db2-5e6b-4ffe-b2b3-d715eed58293 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:51.087965 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f27db2-5e6b-4ffe-b2b3-d715eed58293 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:51.089451 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f27db2-5e6b-4ffe-b2b3-d715eed58293 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:51.091149 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 23/45] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:58:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:58:51.097330 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934e4c5-61e1-4de7-9d68-82d151b94193 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:51.212184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934e4c5-61e1-4de7-9d68-82d151b94193 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:51.246669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd24637-c111-4e8f-879c-cf262f5ddd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:51.247347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd24637-c111-4e8f-879c-cf262f5ddd3d None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:51.247640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd24637-c111-4e8f-879c-cf262f5ddd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:51.247917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd24637-c111-4e8f-879c-cf262f5ddd3d None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:51.248630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd24637-c111-4e8f-879c-cf262f5ddd3d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:51.250721 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd24637-c111-4e8f-879c-cf262f5ddd3d None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:51.278361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 23/46] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jul 20 08:58:51 2022] POST /identity/v3/projects => generated 312 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:58:52.717607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d085a-4416-4153-9f6a-58c8a70a9dca None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:52.718117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d085a-4416-4153-9f6a-58c8a70a9dca None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:52.718198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d085a-4416-4153-9f6a-58c8a70a9dca None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:52.720246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 24/47] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:58:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:58:52.725498 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69068158-7491-401d-a807-7c11b7c1bbb4 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:52.725917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69068158-7491-401d-a807-7c11b7c1bbb4 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:52.726206 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69068158-7491-401d-a807-7c11b7c1bbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:52.743846 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-69068158-7491-401d-a807-7c11b7c1bbb4 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:58:52.745082 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-69068158-7491-401d-a807-7c11b7c1bbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:58:52.758528 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:52.813901 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 24/48] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:58:52.836779 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d085a-4416-4153-9f6a-58c8a70a9dca None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:52.840698 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:52.937763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d085a-4416-4153-9f6a-58c8a70a9dca None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:52.971637 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae07e4-a48c-465e-8ea2-8b5d3845dc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:52.972430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae07e4-a48c-465e-8ea2-8b5d3845dc93 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:52.972719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae07e4-a48c-465e-8ea2-8b5d3845dc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:52.973003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae07e4-a48c-465e-8ea2-8b5d3845dc93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:52.984062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae07e4-a48c-465e-8ea2-8b5d3845dc93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:58:52.993509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-bbae07e4-a48c-465e-8ea2-8b5d3845dc93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:58:52.993509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:58:52.993509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:58:52.993509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:58:52.993509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:58:52.993509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:58:52.993509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:58:52.993509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:58:52.993509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:58:52.993509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:58:52.993509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:58:52.993509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:58:52.993509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:58:52.993509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:58:52.993509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:58:52.993509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:58:52.994371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:58:52.994371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:58:52.994371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:58:52.994371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:58:52.994371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:58:52.994371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:58:52.994371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:58:52.994371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:58:52.994371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:58:52.994371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:58:52.994371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:58:52.994371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:58:52.994371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:58:52.994371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:58:52.994371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:58:52.994371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:58:52.995620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:58:52.995620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:58:52.995620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:58:52.995620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:58:52.995620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:58:52.995620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:58:52.995620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:58:52.995620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:58:52.995620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:58:52.995620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:58:52.995620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:58:52.995620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:58:52.995620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 25/49] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 08:58:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:58:53.001133 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69068158-7491-401d-a807-7c11b7c1bbb4 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:53.098236 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69068158-7491-401d-a807-7c11b7c1bbb4 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:53.130400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4da6c-770f-4e13-b3bf-655b9147c5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:53.131100 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4da6c-770f-4e13-b3bf-655b9147c5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:53.131387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4da6c-770f-4e13-b3bf-655b9147c5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:53.131667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4da6c-770f-4e13-b3bf-655b9147c5f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:53.132420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4da6c-770f-4e13-b3bf-655b9147c5f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:58:53.132729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4da6c-770f-4e13-b3bf-655b9147c5f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:53.134098 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4da6c-770f-4e13-b3bf-655b9147c5f1 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:53.143305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 25/50] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:58:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:58:53.148844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae07e4-a48c-465e-8ea2-8b5d3845dc93 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:53.211695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:53.249574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae07e4-a48c-465e-8ea2-8b5d3845dc93 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:53.283007 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfed7e-b2e2-4fc3-be14-3e637d8539c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:53.283879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfed7e-b2e2-4fc3-be14-3e637d8539c1 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:53.284162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfed7e-b2e2-4fc3-be14-3e637d8539c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:53.284440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfed7e-b2e2-4fc3-be14-3e637d8539c1 None admin] PATH_INFO: `/v3/users/d01511dd98c7416fb8387a8557165556` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:53.285626 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfed7e-b2e2-4fc3-be14-3e637d8539c1 None admin] RBAC: Authorizing `identity:get_user(user_id=d01511dd98c7416fb8387a8557165556)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:53.287103 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfed7e-b2e2-4fc3-be14-3e637d8539c1 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:53.288727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 26/51] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:58:53 2022] GET /identity/v3/users/d01511dd98c7416fb8387a8557165556 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:58:53.294107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4da6c-770f-4e13-b3bf-655b9147c5f1 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:53.307214 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:53.321691 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2241c8-0b60-43f1-9273-7d450759871f None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:53.321977 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2241c8-0b60-43f1-9273-7d450759871f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:53.322259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2241c8-0b60-43f1-9273-7d450759871f None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:53.323657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 27/52] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:58:53.329116 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6ec4f-4811-4614-bcd3-da21e87dfd6b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:53.329416 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6ec4f-4811-4614-bcd3-da21e87dfd6b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:53.329698 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6ec4f-4811-4614-bcd3-da21e87dfd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:53.346696 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-18c6ec4f-4811-4614-bcd3-da21e87dfd6b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:58:53.347895 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-18c6ec4f-4811-4614-bcd3-da21e87dfd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:58:53.398387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4da6c-770f-4e13-b3bf-655b9147c5f1 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:53.437322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb331e07-f44f-4703-a056-a58bebcfbe15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:53.438401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb331e07-f44f-4703-a056-a58bebcfbe15 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:53.438836 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb331e07-f44f-4703-a056-a58bebcfbe15 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:53.439123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb331e07-f44f-4703-a056-a58bebcfbe15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:53.440476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb331e07-f44f-4703-a056-a58bebcfbe15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:53.441962 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb331e07-f44f-4703-a056-a58bebcfbe15 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:53.443954 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 26/53] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:58:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:58:53.450056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb331e07-f44f-4703-a056-a58bebcfbe15 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:53.519752 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 28/54] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:58:53.531563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:53.543291 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6ec4f-4811-4614-bcd3-da21e87dfd6b None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:53.560161 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb331e07-f44f-4703-a056-a58bebcfbe15 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:53.594923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fc57c-0c5d-4c33-9b9e-d1e5b998f653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:53.595585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fc57c-0c5d-4c33-9b9e-d1e5b998f653 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:53.595874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fc57c-0c5d-4c33-9b9e-d1e5b998f653 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:53.596160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fc57c-0c5d-4c33-9b9e-d1e5b998f653 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:53.597000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fc57c-0c5d-4c33-9b9e-d1e5b998f653 None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.domain.id=default, user.id=d01511dd98c7416fb8387a8557165556) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:58:53.597318 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fc57c-0c5d-4c33-9b9e-d1e5b998f653 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:53.598788 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fc57c-0c5d-4c33-9b9e-d1e5b998f653 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:53.604074 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:53.604692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 27/55] 10.222.0.44 () {56 vars in 1322 bytes} [Wed Jul 20 08:58:53 2022] GET /identity/v3/role_assignments?user.id=d01511dd98c7416fb8387a8557165556&scope.domain.id=default&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:58:53.648962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6ec4f-4811-4614-bcd3-da21e87dfd6b None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:53.685196 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48731aef-cd4f-4116-a2ab-93939776ca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:53.686023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48731aef-cd4f-4116-a2ab-93939776ca2f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:53.686313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48731aef-cd4f-4116-a2ab-93939776ca2f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:53.686597 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48731aef-cd4f-4116-a2ab-93939776ca2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:53.688628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48731aef-cd4f-4116-a2ab-93939776ca2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:53.690111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48731aef-cd4f-4116-a2ab-93939776ca2f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:53.691825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 29/56] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:58:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:58:53.697864 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fc57c-0c5d-4c33-9b9e-d1e5b998f653 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:53.801119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fc57c-0c5d-4c33-9b9e-d1e5b998f653 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:53.835243 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4dad23-2944-4273-ae0f-35b4a49c0f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:53.835912 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dad23-2944-4273-ae0f-35b4a49c0f78 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:53.836203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dad23-2944-4273-ae0f-35b4a49c0f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:53.836487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dad23-2944-4273-ae0f-35b4a49c0f78 None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:53.837216 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4dad23-2944-4273-ae0f-35b4a49c0f78 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:53.838716 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4dad23-2944-4273-ae0f-35b4a49c0f78 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:53.851899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:53.863533 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 28/57] 10.222.0.44 () {60 vars in 1140 bytes} [Wed Jul 20 08:58:53 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:58:55.887576 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ef7d8-1de6-4971-bf4a-0517a4928463 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:55.888079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ef7d8-1de6-4971-bf4a-0517a4928463 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:55.888187 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ef7d8-1de6-4971-bf4a-0517a4928463 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:55.889506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 30/58] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:58:55.895106 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457847f-2632-45b1-a86f-034592976923 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:55.895514 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457847f-2632-45b1-a86f-034592976923 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:55.895674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457847f-2632-45b1-a86f-034592976923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:55.912543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-5457847f-2632-45b1-a86f-034592976923 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:58:55.913717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5457847f-2632-45b1-a86f-034592976923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:58:55.978567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 29/59] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:58:56.002298 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ef7d8-1de6-4971-bf4a-0517a4928463 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:56.101496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ef7d8-1de6-4971-bf4a-0517a4928463 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:56.128278 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:56.134246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664250ba-18ed-4fc2-9b64-0bb431a362f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:56.135059 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664250ba-18ed-4fc2-9b64-0bb431a362f2 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:56.135352 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664250ba-18ed-4fc2-9b64-0bb431a362f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:56.135639 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664250ba-18ed-4fc2-9b64-0bb431a362f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:56.136821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664250ba-18ed-4fc2-9b64-0bb431a362f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:56.138315 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664250ba-18ed-4fc2-9b64-0bb431a362f2 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:56.139917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 31/60] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:58:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:58:56.145829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457847f-2632-45b1-a86f-034592976923 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:56.241564 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457847f-2632-45b1-a86f-034592976923 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:56.268142 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:56.279481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4259b-2ec1-483e-85fe-8e1fe2b941e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:56.280157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4259b-2ec1-483e-85fe-8e1fe2b941e2 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:56.280448 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4259b-2ec1-483e-85fe-8e1fe2b941e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:56.280733 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4259b-2ec1-483e-85fe-8e1fe2b941e2 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:56.281589 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4259b-2ec1-483e-85fe-8e1fe2b941e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:56.283082 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4259b-2ec1-483e-85fe-8e1fe2b941e2 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:56.307745 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 30/61] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 08:58:56 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:58:59.377935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc24b26-7375-46e1-be5a-f8b78c237d33 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:59.379005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc24b26-7375-46e1-be5a-f8b78c237d33 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:59.379497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc24b26-7375-46e1-be5a-f8b78c237d33 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:59.381076 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 32/62] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:58:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:58:59.386577 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036837b-c1ef-4e73-8e26-b85fd96d04b7 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:59.387437 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036837b-c1ef-4e73-8e26-b85fd96d04b7 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:59.387929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036837b-c1ef-4e73-8e26-b85fd96d04b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:59.414828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-d036837b-c1ef-4e73-8e26-b85fd96d04b7 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:58:59.416390 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d036837b-c1ef-4e73-8e26-b85fd96d04b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:58:59.509391 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 31/63] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:58:59.553021 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc24b26-7375-46e1-be5a-f8b78c237d33 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:59.588416 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709bf36-4c03-47e6-b174-62a307233d25 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:59.589064 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709bf36-4c03-47e6-b174-62a307233d25 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:59.589660 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709bf36-4c03-47e6-b174-62a307233d25 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:59.591451 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 32/64] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:58:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:58:59.598577 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020fa7d-c503-4b08-a405-6f5712b17498 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:59.599160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020fa7d-c503-4b08-a405-6f5712b17498 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:59.599638 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020fa7d-c503-4b08-a405-6f5712b17498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:59.619142 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-d020fa7d-c503-4b08-a405-6f5712b17498 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:58:59.620663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d020fa7d-c503-4b08-a405-6f5712b17498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:58:59.714669 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc24b26-7375-46e1-be5a-f8b78c237d33 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:59.737026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 33/65] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:58:59.758883 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43541c-da21-4c5b-a905-3cd60b1cadf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:59.759900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43541c-da21-4c5b-a905-3cd60b1cadf2 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:59.760332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43541c-da21-4c5b-a905-3cd60b1cadf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:59.760783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43541c-da21-4c5b-a905-3cd60b1cadf2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:59.773127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43541c-da21-4c5b-a905-3cd60b1cadf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:58:59.774882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020fa7d-c503-4b08-a405-6f5712b17498 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:59.792558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-4f43541c-da21-4c5b-a905-3cd60b1cadf2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:58:59.792558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:58:59.792558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:58:59.792558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:58:59.792558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:58:59.792558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:58:59.792558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:58:59.792558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:58:59.792558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:58:59.792558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:58:59.792558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:58:59.792558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:58:59.792558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:58:59.792558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:58:59.792558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:58:59.792558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:58:59.793646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:58:59.793646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:58:59.793646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:58:59.793646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:58:59.793646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:58:59.793646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:58:59.793646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:58:59.793646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:58:59.793646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:58:59.793646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:58:59.793646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:58:59.793646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:58:59.793646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:58:59.793646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:58:59.793646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:58:59.793646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:58:59.794663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:58:59.794663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:58:59.794663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:58:59.794663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:58:59.794663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:58:59.794663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:58:59.794663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:58:59.794663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:58:59.794663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:58:59.794663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:58:59.794663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:58:59.794663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:58:59.800098 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 33/66] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:58:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:58:59.801584 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6f219-342b-4d32-9368-12eeac44c01c None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:59.802039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6f219-342b-4d32-9368-12eeac44c01c None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:59.807028 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6f219-342b-4d32-9368-12eeac44c01c None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:59.808790 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 34/67] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:58:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:58:59.809665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6f219-342b-4d32-9368-12eeac44c01c None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:58:59.838026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:59.887031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:58:59.921406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020fa7d-c503-4b08-a405-6f5712b17498 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:59.933947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6f219-342b-4d32-9368-12eeac44c01c None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:58:59.972028 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369b9a5-ec5a-4371-bf55-64b671c4d50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:59.973016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369b9a5-ec5a-4371-bf55-64b671c4d50d None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:59.973502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369b9a5-ec5a-4371-bf55-64b671c4d50d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:59.973992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369b9a5-ec5a-4371-bf55-64b671c4d50d None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:59.974987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369b9a5-ec5a-4371-bf55-64b671c4d50d None admin] RBAC: Adding query filter params (name=member) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:58:59.975491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369b9a5-ec5a-4371-bf55-64b671c4d50d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:58:59.975876 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc858a9-0f94-4b45-aecc-0de02c20fc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:58:59.976683 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc858a9-0f94-4b45-aecc-0de02c20fc48 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:59.976970 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc858a9-0f94-4b45-aecc-0de02c20fc48 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:59.977251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc858a9-0f94-4b45-aecc-0de02c20fc48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:58:59.977946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369b9a5-ec5a-4371-bf55-64b671c4d50d None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:58:59.991157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc858a9-0f94-4b45-aecc-0de02c20fc48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:58:59.991790 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 35/68] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:58:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:58:59.993266 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf297880-3553-4cc1-b190-8b05fae7347b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:58:59.993752 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf297880-3553-4cc1-b190-8b05fae7347b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:58:59.994243 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf297880-3553-4cc1-b190-8b05fae7347b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:00.005605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-ccc858a9-0f94-4b45-aecc-0de02c20fc48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:00.005605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:00.005605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:00.005605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:00.005605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:00.005605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:00.005605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:00.005605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:00.005605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:00.005605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:00.005605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:00.005605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:00.005605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:00.005605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:00.005605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:00.005605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:00.006511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:00.006511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:00.006511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:00.006511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:00.006511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:00.006511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:00.006511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:00.006511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:00.006511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:00.006511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:00.006511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:00.006511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:00.006511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:00.006511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:00.006511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:00.006511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:00.007817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:00.007817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:00.007817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:00.007817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:00.007817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:00.007817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:00.007817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:00.007817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:00.007817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:00.007817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:00.007817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:00.007817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:00.007817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 34/69] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 08:58:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:00.007817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc858a9-0f94-4b45-aecc-0de02c20fc48 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:00.016772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-cf297880-3553-4cc1-b190-8b05fae7347b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:00.018226 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-cf297880-3553-4cc1-b190-8b05fae7347b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:00.092212 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 36/70] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:00.093301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf297880-3553-4cc1-b190-8b05fae7347b None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:00.116824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:00.119989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc858a9-0f94-4b45-aecc-0de02c20fc48 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:00.144433 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:00.155289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd583a1-c642-4242-ba1c-7989940dc32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:00.156210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd583a1-c642-4242-ba1c-7989940dc32f None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:00.156485 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd583a1-c642-4242-ba1c-7989940dc32f None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:00.156777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd583a1-c642-4242-ba1c-7989940dc32f None admin] PATH_INFO: `/v3/users/7e0b262e8410401e8d9297ca2732da45` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:00.170084 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd583a1-c642-4242-ba1c-7989940dc32f None admin] RBAC: Authorizing `identity:get_user(user_id=7e0b262e8410401e8d9297ca2732da45)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:00.171598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd583a1-c642-4242-ba1c-7989940dc32f None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:00.173885 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 35/71] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:00 2022] GET /identity/v3/users/7e0b262e8410401e8d9297ca2732da45 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:00.175139 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fcd42-c6ca-4d00-ab4d-6df62ae41357 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:00.175429 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fcd42-c6ca-4d00-ab4d-6df62ae41357 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:00.175712 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fcd42-c6ca-4d00-ab4d-6df62ae41357 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:00.176956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 36/72] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:00.178024 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecf253-19dc-40a8-86b6-b260ef97467e None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:00.178313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecf253-19dc-40a8-86b6-b260ef97467e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:00.178596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecf253-19dc-40a8-86b6-b260ef97467e None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:00.179864 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 37/73] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:00.180630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecf253-19dc-40a8-86b6-b260ef97467e None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:00.196167 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf297880-3553-4cc1-b190-8b05fae7347b None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:00.231513 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a43cb-1206-4f87-bc21-5cd475a28e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:00.232114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a43cb-1206-4f87-bc21-5cd475a28e55 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:00.232406 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a43cb-1206-4f87-bc21-5cd475a28e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:00.232687 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a43cb-1206-4f87-bc21-5cd475a28e55 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:00.233447 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a43cb-1206-4f87-bc21-5cd475a28e55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:00.233759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a43cb-1206-4f87-bc21-5cd475a28e55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:00.235398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a43cb-1206-4f87-bc21-5cd475a28e55 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:00.243428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 37/74] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:00.244404 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a43cb-1206-4f87-bc21-5cd475a28e55 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:00.289025 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecf253-19dc-40a8-86b6-b260ef97467e None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:00.323742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870c4aa-b813-4487-aa2d-52a3d673bd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:00.324553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870c4aa-b813-4487-aa2d-52a3d673bd1e None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:00.324839 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870c4aa-b813-4487-aa2d-52a3d673bd1e None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:00.325118 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870c4aa-b813-4487-aa2d-52a3d673bd1e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:00.335562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870c4aa-b813-4487-aa2d-52a3d673bd1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:00.343512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-6870c4aa-b813-4487-aa2d-52a3d673bd1e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:00.343512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:00.343512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:00.343512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:00.343512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:00.343512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:00.343512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:00.343512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:00.343512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:00.343512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:00.343512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:00.343512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:00.343512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:00.343512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:00.343512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:00.343512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:00.344489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:00.344489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:00.344489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:00.344489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:00.344489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:00.344489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:00.344489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:00.344489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:00.344489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:00.344489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:00.344489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:00.344489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:00.344489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:00.344489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:00.344489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:00.344489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:00.345440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:00.345440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:00.345440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:00.345440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:00.345440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:00.345440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:00.345440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:00.345440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:00.345440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:00.345440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:00.345440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:00.345440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:00.345440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 38/75] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 08:59:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:00.347960 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cd094-e139-4ed6-847c-a87f071b6ec4 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:00.348251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cd094-e139-4ed6-847c-a87f071b6ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:00.348542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cd094-e139-4ed6-847c-a87f071b6ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:00.351348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a43cb-1206-4f87-bc21-5cd475a28e55 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:00.364679 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:00.365160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-894cd094-e139-4ed6-847c-a87f071b6ec4 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:00.366363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-894cd094-e139-4ed6-847c-a87f071b6ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:00.380149 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:00.389470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef9ede-650e-4692-abbb-efb1b8bd1e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:00.390133 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef9ede-650e-4692-abbb-efb1b8bd1e0c None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:00.390469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef9ede-650e-4692-abbb-efb1b8bd1e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:00.390785 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef9ede-650e-4692-abbb-efb1b8bd1e0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:00.435336 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 39/76] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:00.436682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e7642-45eb-4739-af55-d8d2c1ab1bbe None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:00.436981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e7642-45eb-4739-af55-d8d2c1ab1bbe None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:00.437281 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e7642-45eb-4739-af55-d8d2c1ab1bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:00.463703 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef9ede-650e-4692-abbb-efb1b8bd1e0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:00.465166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef9ede-650e-4692-abbb-efb1b8bd1e0c None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:00.465700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-9e8e7642-45eb-4739-af55-d8d2c1ab1bbe None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:00.466933 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-9e8e7642-45eb-4739-af55-d8d2c1ab1bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:00.534107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 40/77] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:00.535072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e7642-45eb-4739-af55-d8d2c1ab1bbe None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:00.568525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 38/78] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:00.569748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef9ede-650e-4692-abbb-efb1b8bd1e0c None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:00.587399 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:00.630282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:00.655612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e7642-45eb-4739-af55-d8d2c1ab1bbe None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:00.676208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef9ede-650e-4692-abbb-efb1b8bd1e0c None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:00.689430 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30842c41-f445-4e8a-91e5-b04ec24633b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:00.690298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30842c41-f445-4e8a-91e5-b04ec24633b3 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:00.690580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30842c41-f445-4e8a-91e5-b04ec24633b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:00.690888 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30842c41-f445-4e8a-91e5-b04ec24633b3 None admin] PATH_INFO: `/v3/users/d01511dd98c7416fb8387a8557165556` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:00.692065 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30842c41-f445-4e8a-91e5-b04ec24633b3 None admin] RBAC: Authorizing `identity:get_user(user_id=d01511dd98c7416fb8387a8557165556)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:00.693518 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30842c41-f445-4e8a-91e5-b04ec24633b3 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:00.695424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 41/79] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:00 2022] GET /identity/v3/users/d01511dd98c7416fb8387a8557165556 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:00.696177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30842c41-f445-4e8a-91e5-b04ec24633b3 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:00.713788 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6e658-22a1-4624-8ad8-599c2048b53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:00.714461 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6e658-22a1-4624-8ad8-599c2048b53a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:00.714792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6e658-22a1-4624-8ad8-599c2048b53a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:00.715080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6e658-22a1-4624-8ad8-599c2048b53a None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:00.715835 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6e658-22a1-4624-8ad8-599c2048b53a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:00.716145 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6e658-22a1-4624-8ad8-599c2048b53a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:00.717518 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6e658-22a1-4624-8ad8-599c2048b53a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:00.726435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 39/80] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:00.727208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6e658-22a1-4624-8ad8-599c2048b53a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:00.795182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30842c41-f445-4e8a-91e5-b04ec24633b3 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:00.804078 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:00.829610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5160f7b-717c-43b4-94c1-b386c52ef103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:00.830344 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6e658-22a1-4624-8ad8-599c2048b53a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:00.830442 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5160f7b-717c-43b4-94c1-b386c52ef103 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:00.830729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5160f7b-717c-43b4-94c1-b386c52ef103 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:00.831015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5160f7b-717c-43b4-94c1-b386c52ef103 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:00.832191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5160f7b-717c-43b4-94c1-b386c52ef103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:00.833639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5160f7b-717c-43b4-94c1-b386c52ef103 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:00.835283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 42/81] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:00.836047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5160f7b-717c-43b4-94c1-b386c52ef103 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:00.853027 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:00.864969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfad3f-25b8-46c8-9e41-291cd0063957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:00.865767 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfad3f-25b8-46c8-9e41-291cd0063957 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:00.866056 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfad3f-25b8-46c8-9e41-291cd0063957 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:00.866338 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfad3f-25b8-46c8-9e41-291cd0063957 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:00.876366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfad3f-25b8-46c8-9e41-291cd0063957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:00.883952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-0edfad3f-25b8-46c8-9e41-291cd0063957 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:00.883952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:00.883952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:00.883952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:00.883952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:00.883952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:00.883952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:00.883952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:00.883952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:00.883952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:00.883952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:00.883952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:00.883952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:00.883952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:00.883952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:00.883952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:00.884789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:00.884789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:00.884789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:00.884789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:00.884789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:00.884789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:00.884789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:00.884789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:00.884789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:00.884789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:00.884789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:00.884789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:00.884789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:00.884789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:00.884789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:00.884789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:00.885565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:00.885565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:00.885565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:00.885565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:00.885565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:00.885565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:00.885565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:00.885565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:00.885565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:00.885565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:00.885565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:00.885565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:00.885565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 40/82] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:00.886191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfad3f-25b8-46c8-9e41-291cd0063957 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:00.940402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5160f7b-717c-43b4-94c1-b386c52ef103 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:00.974385 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a26d2-2955-4cf8-afc2-e42fd2929f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:00.975280 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a26d2-2955-4cf8-afc2-e42fd2929f24 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:00.975569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a26d2-2955-4cf8-afc2-e42fd2929f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:00.975850 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a26d2-2955-4cf8-afc2-e42fd2929f24 None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:00.977042 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a26d2-2955-4cf8-afc2-e42fd2929f24 None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:00.978480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a26d2-2955-4cf8-afc2-e42fd2929f24 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:00.980201 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 43/83] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:00 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:00.980956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a26d2-2955-4cf8-afc2-e42fd2929f24 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:00.985525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfad3f-25b8-46c8-9e41-291cd0063957 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:01.019816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed9790-6080-444d-8705-88c8a4ebcc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:01.020487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed9790-6080-444d-8705-88c8a4ebcc6b None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:01.020770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed9790-6080-444d-8705-88c8a4ebcc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:01.021047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed9790-6080-444d-8705-88c8a4ebcc6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:01.023680 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:01.065463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:01.084083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a26d2-2955-4cf8-afc2-e42fd2929f24 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:01.091547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed9790-6080-444d-8705-88c8a4ebcc6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:01.093057 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed9790-6080-444d-8705-88c8a4ebcc6b None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:01.117605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345d3fd-16ea-41ca-9586-b19aee341ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:01.118566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345d3fd-16ea-41ca-9586-b19aee341ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:01.118887 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345d3fd-16ea-41ca-9586-b19aee341ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:01.119167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345d3fd-16ea-41ca-9586-b19aee341ba3 None admin] PATH_INFO: `/v3/users/7e0b262e8410401e8d9297ca2732da45` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:01.120349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345d3fd-16ea-41ca-9586-b19aee341ba3 None admin] RBAC: Authorizing `identity:get_user(user_id=7e0b262e8410401e8d9297ca2732da45)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:01.121786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345d3fd-16ea-41ca-9586-b19aee341ba3 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:01.123435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 44/84] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:00 2022] GET /identity/v3/users/7e0b262e8410401e8d9297ca2732da45 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:01.124165 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345d3fd-16ea-41ca-9586-b19aee341ba3 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:01.191422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 41/85] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:01.192269 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed9790-6080-444d-8705-88c8a4ebcc6b None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:01.224577 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345d3fd-16ea-41ca-9586-b19aee341ba3 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:01.224736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:01.259796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcc5fb-93a4-467b-b89f-4ea186b10948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:01.260472 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcc5fb-93a4-467b-b89f-4ea186b10948 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:01.260761 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcc5fb-93a4-467b-b89f-4ea186b10948 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:01.261042 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcc5fb-93a4-467b-b89f-4ea186b10948 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:01.261765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dcc5fb-93a4-467b-b89f-4ea186b10948 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:01.263253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dcc5fb-93a4-467b-b89f-4ea186b10948 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:01.282810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 45/86] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 08:59:01 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:59:01.283690 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcc5fb-93a4-467b-b89f-4ea186b10948 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:01.293323 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed9790-6080-444d-8705-88c8a4ebcc6b None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:01.293748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:01.336012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be480b5-e38b-4a37-9224-bfd2c274b09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:01.336622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be480b5-e38b-4a37-9224-bfd2c274b09d None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:01.336909 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be480b5-e38b-4a37-9224-bfd2c274b09d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:01.337197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be480b5-e38b-4a37-9224-bfd2c274b09d None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:01.338165 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be480b5-e38b-4a37-9224-bfd2c274b09d None admin] RBAC: Adding query filter params (name=member) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:01.338285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be480b5-e38b-4a37-9224-bfd2c274b09d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:01.339748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be480b5-e38b-4a37-9224-bfd2c274b09d None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:01.349390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 42/87] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:01.350271 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be480b5-e38b-4a37-9224-bfd2c274b09d None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:01.416149 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcc5fb-93a4-467b-b89f-4ea186b10948 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:01.458274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:01.461219 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71c5da-f44a-48a7-b1c2-492e38a2a99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:01.462024 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71c5da-f44a-48a7-b1c2-492e38a2a99b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:01.462317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71c5da-f44a-48a7-b1c2-492e38a2a99b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:01.462602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71c5da-f44a-48a7-b1c2-492e38a2a99b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:01.463539 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71c5da-f44a-48a7-b1c2-492e38a2a99b None admin] RBAC: Adding query filter params (role.id=0e2011aac3e1430cb96bdbf810e89eb7, scope.project.id=81fa154d1fc04266abee9f0051cba2fa, user.id=7e0b262e8410401e8d9297ca2732da45) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:01.463950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71c5da-f44a-48a7-b1c2-492e38a2a99b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:01.465316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71c5da-f44a-48a7-b1c2-492e38a2a99b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:01.466144 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be480b5-e38b-4a37-9224-bfd2c274b09d None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:01.471496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 46/88] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:01 2022] GET /identity/v3/role_assignments?user.id=7e0b262e8410401e8d9297ca2732da45&scope.project.id=81fa154d1fc04266abee9f0051cba2fa&role.id=0e2011aac3e1430cb96bdbf810e89eb7 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:01.472145 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71c5da-f44a-48a7-b1c2-492e38a2a99b None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:01.519096 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74df5c-cea1-4808-83f7-f873f40a5115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:01.519809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74df5c-cea1-4808-83f7-f873f40a5115 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:01.520197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74df5c-cea1-4808-83f7-f873f40a5115 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:01.520384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74df5c-cea1-4808-83f7-f873f40a5115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:01.578507 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:01.625604 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71c5da-f44a-48a7-b1c2-492e38a2a99b None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:01.639624 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74df5c-cea1-4808-83f7-f873f40a5115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:01.642644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74df5c-cea1-4808-83f7-f873f40a5115 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:01.674853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c3c0f-1250-4118-8d66-fa8aca0cb96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:01.675734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c3c0f-1250-4118-8d66-fa8aca0cb96e None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:01.676022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c3c0f-1250-4118-8d66-fa8aca0cb96e None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:01.676300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c3c0f-1250-4118-8d66-fa8aca0cb96e None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:01.678357 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c3c0f-1250-4118-8d66-fa8aca0cb96e None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:01.679997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c3c0f-1250-4118-8d66-fa8aca0cb96e None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:01.681937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 47/89] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:01 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:01.682487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c3c0f-1250-4118-8d66-fa8aca0cb96e None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:01.762154 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:01.771186 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 43/90] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:01.772337 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74df5c-cea1-4808-83f7-f873f40a5115 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:01.807318 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c3c0f-1250-4118-8d66-fa8aca0cb96e None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:01.842953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c36f6-520b-470f-b910-fb839adf78fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:01.844440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c36f6-520b-470f-b910-fb839adf78fa None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:01.844775 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c36f6-520b-470f-b910-fb839adf78fa None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:01.845062 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c36f6-520b-470f-b910-fb839adf78fa None admin] PATH_INFO: `/v3/users/7e0b262e8410401e8d9297ca2732da45` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:01.846324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c36f6-520b-470f-b910-fb839adf78fa None admin] RBAC: Authorizing `identity:get_user(user_id=7e0b262e8410401e8d9297ca2732da45)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:01.847810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c36f6-520b-470f-b910-fb839adf78fa None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:01.849749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 48/91] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:01 2022] GET /identity/v3/users/7e0b262e8410401e8d9297ca2732da45 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:01.850596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c36f6-520b-470f-b910-fb839adf78fa None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:01.869016 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:01.924632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74df5c-cea1-4808-83f7-f873f40a5115 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:01.988988 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f45f96-72c7-479f-a4eb-159ce68ff70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:01.989713 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f45f96-72c7-479f-a4eb-159ce68ff70f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:01.989996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f45f96-72c7-479f-a4eb-159ce68ff70f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:01.990306 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f45f96-72c7-479f-a4eb-159ce68ff70f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:01.991205 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f45f96-72c7-479f-a4eb-159ce68ff70f None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=81fa154d1fc04266abee9f0051cba2fa, user.id=d01511dd98c7416fb8387a8557165556) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:01.991571 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f45f96-72c7-479f-a4eb-159ce68ff70f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:01.993017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f45f96-72c7-479f-a4eb-159ce68ff70f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:02.001582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 44/92] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:01 2022] GET /identity/v3/role_assignments?user.id=d01511dd98c7416fb8387a8557165556&scope.project.id=81fa154d1fc04266abee9f0051cba2fa&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:02.002418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f45f96-72c7-479f-a4eb-159ce68ff70f None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:02.045918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c36f6-520b-470f-b910-fb839adf78fa None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:02.104420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:02.134729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141bf17e-72cb-481f-8802-431804eedcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:02.135635 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bf17e-72cb-481f-8802-431804eedcb4 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:02.135920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bf17e-72cb-481f-8802-431804eedcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:02.136306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bf17e-72cb-481f-8802-431804eedcb4 None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:02.137651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141bf17e-72cb-481f-8802-431804eedcb4 None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:02.139256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141bf17e-72cb-481f-8802-431804eedcb4 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:02.142084 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 49/93] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:01 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:02.148498 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141bf17e-72cb-481f-8802-431804eedcb4 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:02.179257 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f45f96-72c7-479f-a4eb-159ce68ff70f None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:02.261240 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859a849-a5b1-4b1f-bc11-b2e351454b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:02.262264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859a849-a5b1-4b1f-bc11-b2e351454b37 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:02.262777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859a849-a5b1-4b1f-bc11-b2e351454b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:02.263280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859a849-a5b1-4b1f-bc11-b2e351454b37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:02.303783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:02.355852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141bf17e-72cb-481f-8802-431804eedcb4 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:02.410564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859a849-a5b1-4b1f-bc11-b2e351454b37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:02.412404 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859a849-a5b1-4b1f-bc11-b2e351454b37 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:02.431534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ada759-7f76-40e8-99b4-71bd38146ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:02.432140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ada759-7f76-40e8-99b4-71bd38146ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:02.432426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ada759-7f76-40e8-99b4-71bd38146ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:02.432706 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ada759-7f76-40e8-99b4-71bd38146ce4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:02.433551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ada759-7f76-40e8-99b4-71bd38146ce4 None admin] RBAC: Adding query filter params (role.id=05ca4843c34e463c94de09f756d95318, scope.project.id=81fa154d1fc04266abee9f0051cba2fa, user.id=7e0b262e8410401e8d9297ca2732da45) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:02.433871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ada759-7f76-40e8-99b4-71bd38146ce4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:02.435366 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ada759-7f76-40e8-99b4-71bd38146ce4 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:02.452676 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 50/94] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:02 2022] GET /identity/v3/role_assignments?user.id=7e0b262e8410401e8d9297ca2732da45&scope.project.id=81fa154d1fc04266abee9f0051cba2fa&role.id=05ca4843c34e463c94de09f756d95318 => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:02.534161 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:02.606745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 45/95] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:02.613000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ada759-7f76-40e8-99b4-71bd38146ce4 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:02.769107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ada759-7f76-40e8-99b4-71bd38146ce4 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:02.820598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e17d4-08da-4a40-87fd-b969be202f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:02.821475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e17d4-08da-4a40-87fd-b969be202f5d None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:02.821762 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e17d4-08da-4a40-87fd-b969be202f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:02.822043 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e17d4-08da-4a40-87fd-b969be202f5d None admin] PATH_INFO: `/v3/projects/3366828c9f264c46a2d21ee94c2e705d` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:02.823872 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e17d4-08da-4a40-87fd-b969be202f5d None admin] RBAC: Authorizing `identity:get_project(project_id=3366828c9f264c46a2d21ee94c2e705d)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:02.825381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e17d4-08da-4a40-87fd-b969be202f5d None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:02.827283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 51/96] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:02 2022] GET /identity/v3/projects/3366828c9f264c46a2d21ee94c2e705d => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:02.835751 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859a849-a5b1-4b1f-bc11-b2e351454b37 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:02.985851 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859a849-a5b1-4b1f-bc11-b2e351454b37 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:03.026511 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94eb93-f754-4df3-99d7-97c62a40737a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:03.027545 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94eb93-f754-4df3-99d7-97c62a40737a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:03.028031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94eb93-f754-4df3-99d7-97c62a40737a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:03.028507 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94eb93-f754-4df3-99d7-97c62a40737a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:03.029561 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94eb93-f754-4df3-99d7-97c62a40737a None admin] RBAC: Adding query filter params (role.id=0e2011aac3e1430cb96bdbf810e89eb7, scope.project.id=3366828c9f264c46a2d21ee94c2e705d, user.id=7e0b262e8410401e8d9297ca2732da45) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:03.030069 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94eb93-f754-4df3-99d7-97c62a40737a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:03.032620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94eb93-f754-4df3-99d7-97c62a40737a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:03.042796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 46/97] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:02 2022] GET /identity/v3/role_assignments?user.id=7e0b262e8410401e8d9297ca2732da45&scope.project.id=3366828c9f264c46a2d21ee94c2e705d&role.id=0e2011aac3e1430cb96bdbf810e89eb7 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:04.125634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123ec22-f63c-40a8-a1f1-f23bc5d24a94 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:04.126142 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123ec22-f63c-40a8-a1f1-f23bc5d24a94 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:04.126216 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123ec22-f63c-40a8-a1f1-f23bc5d24a94 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:04.127706 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 52/98] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:04.133182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e21cca-63b0-47cb-b5d8-c38fcf5151d6 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:04.133618 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e21cca-63b0-47cb-b5d8-c38fcf5151d6 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:04.133789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e21cca-63b0-47cb-b5d8-c38fcf5151d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:04.150887 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-d1e21cca-63b0-47cb-b5d8-c38fcf5151d6 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:04.152064 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d1e21cca-63b0-47cb-b5d8-c38fcf5151d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:04.170407 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:04.227476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 47/99] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:04.250328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123ec22-f63c-40a8-a1f1-f23bc5d24a94 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:04.259953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:04.384935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123ec22-f63c-40a8-a1f1-f23bc5d24a94 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:04.436747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c7a1f-34e2-4370-85eb-223ac7d71910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:04.438120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c7a1f-34e2-4370-85eb-223ac7d71910 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:04.438406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c7a1f-34e2-4370-85eb-223ac7d71910 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:04.438703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c7a1f-34e2-4370-85eb-223ac7d71910 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:04.456056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59c7a1f-34e2-4370-85eb-223ac7d71910 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:04.463805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-d59c7a1f-34e2-4370-85eb-223ac7d71910 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:04.463805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:04.463805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:04.463805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:04.463805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:04.463805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:04.463805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:04.463805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:04.463805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:04.463805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:04.463805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:04.463805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:04.463805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:04.463805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:04.463805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:04.463805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:04.464649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:04.464649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:04.464649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:04.464649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:04.464649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:04.464649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:04.464649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:04.464649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:04.464649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:04.464649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:04.464649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:04.464649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:04.464649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:04.464649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:04.464649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:04.464649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:04.465397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:04.465397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:04.465397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:04.465397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:04.465397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:04.465397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:04.465397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:04.465397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:04.465397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:04.465397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:04.465397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:04.465397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:04.465397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 53/100] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:04.471812 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e21cca-63b0-47cb-b5d8-c38fcf5151d6 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:04.654492 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e21cca-63b0-47cb-b5d8-c38fcf5151d6 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:04.691625 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1e4cd-11d0-495e-84a9-c931adf19811 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:04.691913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1e4cd-11d0-495e-84a9-c931adf19811 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:04.692189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1e4cd-11d0-495e-84a9-c931adf19811 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:04.694091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 54/101] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:04.697002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d465a-9b29-4449-bb64-feca81158cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:04.697765 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d465a-9b29-4449-bb64-feca81158cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:04.698063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d465a-9b29-4449-bb64-feca81158cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:04.698345 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d465a-9b29-4449-bb64-feca81158cf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:04.699131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d465a-9b29-4449-bb64-feca81158cf6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:04.699440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d465a-9b29-4449-bb64-feca81158cf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:04.700827 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d465a-9b29-4449-bb64-feca81158cf6 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:04.704666 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea93a0-d41f-47e0-a301-af801ad0d2fa None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:04.704959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea93a0-d41f-47e0-a301-af801ad0d2fa None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:04.705243 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea93a0-d41f-47e0-a301-af801ad0d2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:04.714585 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 48/102] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:04.715894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d465a-9b29-4449-bb64-feca81158cf6 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:04.749287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-43ea93a0-d41f-47e0-a301-af801ad0d2fa None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:04.751426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-43ea93a0-d41f-47e0-a301-af801ad0d2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:04.751936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:04.819221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d465a-9b29-4449-bb64-feca81158cf6 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:04.853941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743cc4f-a9fa-4c10-8f44-becc3c6bc7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:04.855113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743cc4f-a9fa-4c10-8f44-becc3c6bc7aa None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:04.855594 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743cc4f-a9fa-4c10-8f44-becc3c6bc7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:04.856010 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743cc4f-a9fa-4c10-8f44-becc3c6bc7aa None admin] PATH_INFO: `/v3/users/7e0b262e8410401e8d9297ca2732da45` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:04.857435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743cc4f-a9fa-4c10-8f44-becc3c6bc7aa None admin] RBAC: Authorizing `identity:get_user(user_id=7e0b262e8410401e8d9297ca2732da45)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:04.859731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743cc4f-a9fa-4c10-8f44-becc3c6bc7aa None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:04.861646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 49/103] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:04 2022] GET /identity/v3/users/7e0b262e8410401e8d9297ca2732da45 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:04.867429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743cc4f-a9fa-4c10-8f44-becc3c6bc7aa None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:04.924844 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:04.940388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 55/104] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:04.941872 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300850fd-f1a1-4a13-87f8-43a20a05615d None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:04.942170 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300850fd-f1a1-4a13-87f8-43a20a05615d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:04.942456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300850fd-f1a1-4a13-87f8-43a20a05615d None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:04.944917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 56/105] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:04.950418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb81045-258d-4395-8b2c-9f9ecf4e74fc None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:04.950661 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb81045-258d-4395-8b2c-9f9ecf4e74fc None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:04.951001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb81045-258d-4395-8b2c-9f9ecf4e74fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:04.983916 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-fdb81045-258d-4395-8b2c-9f9ecf4e74fc None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:04.986347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-fdb81045-258d-4395-8b2c-9f9ecf4e74fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:04.995364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:05.007244 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743cc4f-a9fa-4c10-8f44-becc3c6bc7aa None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:05.042062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6995e-ad7a-4822-84c1-059932128fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:05.043420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6995e-ad7a-4822-84c1-059932128fdb None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:05.043909 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6995e-ad7a-4822-84c1-059932128fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:05.044360 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6995e-ad7a-4822-84c1-059932128fdb None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:05.045718 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6995e-ad7a-4822-84c1-059932128fdb None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:05.048010 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6995e-ad7a-4822-84c1-059932128fdb None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:05.049895 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 50/106] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:04 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:05.050945 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6995e-ad7a-4822-84c1-059932128fdb None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:05.116857 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 57/107] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:05.118140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb81045-258d-4395-8b2c-9f9ecf4e74fc None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:05.162370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6995e-ad7a-4822-84c1-059932128fdb None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:05.208360 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5e733-6b4c-45b4-b83e-2422c8b3c362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:05.209241 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5e733-6b4c-45b4-b83e-2422c8b3c362 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:05.209534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5e733-6b4c-45b4-b83e-2422c8b3c362 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:05.209813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5e733-6b4c-45b4-b83e-2422c8b3c362 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:05.223747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5e733-6b4c-45b4-b83e-2422c8b3c362 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:05.233411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-59d5e733-6b4c-45b4-b83e-2422c8b3c362 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:05.233411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:05.233411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:05.233411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:05.233411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:05.233411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:05.233411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:05.233411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:05.233411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:05.233411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:05.233411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:05.233411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:05.233411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:05.233411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:05.233411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:05.233411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:05.234470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:05.234470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:05.234470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:05.234470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:05.234470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:05.234470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:05.234470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:05.234470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:05.234470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:05.234470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:05.234470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:05.234470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:05.234470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:05.234470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:05.234470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:05.234470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:05.235286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:05.235286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:05.235286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:05.235286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:05.235286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:05.235286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:05.235286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:05.235286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:05.235286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:05.235286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:05.235286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:05.235286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:05.239234 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 51/108] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:05.239234 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03ea83-b94d-41a2-9e59-46e5d9d891ea None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:05.239234 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03ea83-b94d-41a2-9e59-46e5d9d891ea None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:05.239234 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03ea83-b94d-41a2-9e59-46e5d9d891ea None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:05.241180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 52/109] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:05.243527 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03ea83-b94d-41a2-9e59-46e5d9d891ea None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:05.251647 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb81045-258d-4395-8b2c-9f9ecf4e74fc None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:05.264978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:05.289260 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b26f0-5a94-4fce-8317-c137ec049c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:05.290527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b26f0-5a94-4fce-8317-c137ec049c4e None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:05.291195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b26f0-5a94-4fce-8317-c137ec049c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:05.291670 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b26f0-5a94-4fce-8317-c137ec049c4e None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa/users/7e0b262e8410401e8d9297ca2732da45/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:05.294635 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b26f0-5a94-4fce-8317-c137ec049c4e None admin] RBAC: Authorizing `identity:create_grant(project_id=81fa154d1fc04266abee9f0051cba2fa, user_id=7e0b262e8410401e8d9297ca2732da45, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:05.296448 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b26f0-5a94-4fce-8317-c137ec049c4e None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:05.302915 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:05.317395 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 58/110] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 08:59:05 2022] PUT /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa/users/7e0b262e8410401e8d9297ca2732da45/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:59:05.318634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b26f0-5a94-4fce-8317-c137ec049c4e None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:05.371498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03ea83-b94d-41a2-9e59-46e5d9d891ea None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:05.409984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e26b99-524e-439c-a456-c61331328d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:05.410904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e26b99-524e-439c-a456-c61331328d37 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:05.411191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e26b99-524e-439c-a456-c61331328d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:05.411474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e26b99-524e-439c-a456-c61331328d37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:05.412731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e26b99-524e-439c-a456-c61331328d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:05.414423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e26b99-524e-439c-a456-c61331328d37 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:05.416283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 53/111] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:05.423595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808ca07-ed7e-40a1-9b59-c6ab30507b99 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:05.423780 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808ca07-ed7e-40a1-9b59-c6ab30507b99 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:05.423998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808ca07-ed7e-40a1-9b59-c6ab30507b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:05.442473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-3808ca07-ed7e-40a1-9b59-c6ab30507b99 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:05.444113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-3808ca07-ed7e-40a1-9b59-c6ab30507b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:05.445605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b26f0-5a94-4fce-8317-c137ec049c4e None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:05.485085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9700f5f-b566-48cd-a160-6d736bfeb61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:05.486096 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9700f5f-b566-48cd-a160-6d736bfeb61f None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:05.486601 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9700f5f-b566-48cd-a160-6d736bfeb61f None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:05.487198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9700f5f-b566-48cd-a160-6d736bfeb61f None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:05.488228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9700f5f-b566-48cd-a160-6d736bfeb61f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:05.488739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9700f5f-b566-48cd-a160-6d736bfeb61f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:05.490530 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9700f5f-b566-48cd-a160-6d736bfeb61f None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:05.501987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 59/112] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:05.503243 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9700f5f-b566-48cd-a160-6d736bfeb61f None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:05.522729 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 54/113] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:05.523392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808ca07-ed7e-40a1-9b59-c6ab30507b99 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:05.537659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:05.589996 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:05.655079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808ca07-ed7e-40a1-9b59-c6ab30507b99 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:05.658371 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9700f5f-b566-48cd-a160-6d736bfeb61f None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:05.698957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf43dca-599a-4ff5-ad44-dc66a639e5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:05.699197 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795e683-1d14-4387-bb77-9e55d11a47d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:05.699831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf43dca-599a-4ff5-ad44-dc66a639e5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:05.699951 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795e683-1d14-4387-bb77-9e55d11a47d7 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:05.700156 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf43dca-599a-4ff5-ad44-dc66a639e5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:05.700251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795e683-1d14-4387-bb77-9e55d11a47d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:05.700388 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf43dca-599a-4ff5-ad44-dc66a639e5c5 None admin] PATH_INFO: `/v3/users/07188900e83d448685f5ee569bb0b6da` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:05.700473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795e683-1d14-4387-bb77-9e55d11a47d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:05.701296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795e683-1d14-4387-bb77-9e55d11a47d7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:05.702812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795e683-1d14-4387-bb77-9e55d11a47d7 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:05.719134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf43dca-599a-4ff5-ad44-dc66a639e5c5 None admin] RBAC: Authorizing `identity:get_user(user_id=07188900e83d448685f5ee569bb0b6da)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:05.720936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf43dca-599a-4ff5-ad44-dc66a639e5c5 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:05.723658 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 55/114] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:05 2022] GET /identity/v3/users/07188900e83d448685f5ee569bb0b6da => generated 289 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:05.724073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf43dca-599a-4ff5-ad44-dc66a639e5c5 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:05.731221 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 60/115] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jul 20 08:59:05 2022] POST /identity/v3/projects => generated 302 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:59:05.732974 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8e468-79f4-44a9-8db7-6cb2147d6694 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:05.733470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8e468-79f4-44a9-8db7-6cb2147d6694 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:05.733925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8e468-79f4-44a9-8db7-6cb2147d6694 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:05.736456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 61/116] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:05.737525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8e468-79f4-44a9-8db7-6cb2147d6694 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:05.802259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:05.854049 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8e468-79f4-44a9-8db7-6cb2147d6694 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:05.856657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf43dca-599a-4ff5-ad44-dc66a639e5c5 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:05.867126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:05.893666 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c45c4f-f92b-4b80-858d-d906e1cc52f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:05.894796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c45c4f-f92b-4b80-858d-d906e1cc52f6 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:05.895313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c45c4f-f92b-4b80-858d-d906e1cc52f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:05.895826 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c45c4f-f92b-4b80-858d-d906e1cc52f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:05.936885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df3bd6-a8ec-4e29-8112-8fc5e570e307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:05.938544 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df3bd6-a8ec-4e29-8112-8fc5e570e307 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:05.938898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df3bd6-a8ec-4e29-8112-8fc5e570e307 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:05.939218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df3bd6-a8ec-4e29-8112-8fc5e570e307 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:05.962548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df3bd6-a8ec-4e29-8112-8fc5e570e307 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:05.974442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-49df3bd6-a8ec-4e29-8112-8fc5e570e307 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:05.974442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:05.974442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:05.974442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:05.974442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:05.974442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:05.974442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:05.974442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:05.974442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:05.974442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:05.974442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:05.974442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:05.974442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:05.974442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:05.974442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:05.974442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:05.975534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:05.975534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:05.975534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:05.975534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:05.975534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:05.975534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:05.975534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:05.975534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:05.975534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:05.975534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:05.975534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:05.975534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:05.975534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:05.975534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:05.975534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:05.975534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:05.976299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:05.976299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:05.976299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:05.976299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:05.976299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:05.976299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:05.976299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:05.976299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:05.976299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:05.976299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:05.976299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:05.976299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:05.976299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 56/117] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 08:59:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:05.977927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae379b-419e-4022-82c7-d105e8b72808 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:05.978362 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae379b-419e-4022-82c7-d105e8b72808 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:05.978748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae379b-419e-4022-82c7-d105e8b72808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:05.980422 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c45c4f-f92b-4b80-858d-d906e1cc52f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:05.982351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c45c4f-f92b-4b80-858d-d906e1cc52f6 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:06.002337 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-39ae379b-419e-4022-82c7-d105e8b72808 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:06.003867 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-39ae379b-419e-4022-82c7-d105e8b72808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:06.090345 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 57/118] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:06.093849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd80773-50b2-4f78-98e1-01f92e9ef5ec None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:06.093849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd80773-50b2-4f78-98e1-01f92e9ef5ec None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:06.093849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd80773-50b2-4f78-98e1-01f92e9ef5ec None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:06.093849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 58/119] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:06.094346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd80773-50b2-4f78-98e1-01f92e9ef5ec None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:06.128703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:06.132648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 62/120] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:06.134007 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b290dc1-6ef4-431a-b865-0788220e126b None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:06.134371 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b290dc1-6ef4-431a-b865-0788220e126b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:06.134747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b290dc1-6ef4-431a-b865-0788220e126b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:06.156739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-8b290dc1-6ef4-431a-b865-0788220e126b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:06.158043 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8b290dc1-6ef4-431a-b865-0788220e126b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:06.189469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:06.235332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 63/121] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:06.236384 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b290dc1-6ef4-431a-b865-0788220e126b None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:06.249402 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd80773-50b2-4f78-98e1-01f92e9ef5ec None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:06.299260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febfc60-896f-4e49-9bde-0e76e79ab1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:06.299866 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febfc60-896f-4e49-9bde-0e76e79ab1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:06.300160 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febfc60-896f-4e49-9bde-0e76e79ab1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:06.300446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febfc60-896f-4e49-9bde-0e76e79ab1c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:06.301420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febfc60-896f-4e49-9bde-0e76e79ab1c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:06.301845 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febfc60-896f-4e49-9bde-0e76e79ab1c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:06.303262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febfc60-896f-4e49-9bde-0e76e79ab1c9 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:06.315499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 59/122] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:06.316717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febfc60-896f-4e49-9bde-0e76e79ab1c9 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:06.342356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b290dc1-6ef4-431a-b865-0788220e126b None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:06.373284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:06.381054 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324cc8e-e8d3-49ab-b07a-1a0d8893eded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:06.382074 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324cc8e-e8d3-49ab-b07a-1a0d8893eded None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:06.382435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324cc8e-e8d3-49ab-b07a-1a0d8893eded None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:06.382851 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324cc8e-e8d3-49ab-b07a-1a0d8893eded None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:06.402591 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324cc8e-e8d3-49ab-b07a-1a0d8893eded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:06.413464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-a324cc8e-e8d3-49ab-b07a-1a0d8893eded None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:06.413464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:06.413464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:06.413464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:06.413464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:06.413464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:06.413464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:06.413464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:06.413464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:06.413464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:06.413464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:06.413464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:06.413464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:06.413464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:06.413464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:06.413464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:06.414328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:06.414328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:06.414328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:06.414328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:06.414328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:06.414328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:06.414328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:06.414328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:06.414328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:06.414328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:06.414328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:06.414328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:06.414328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:06.414328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:06.414328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:06.414328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:06.415112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:06.415112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:06.415112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:06.415112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:06.415112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:06.415112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:06.415112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:06.415112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:06.415112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:06.415112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:06.415112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:06.415112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:06.415112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 64/123] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 08:59:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:06.416556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324cc8e-e8d3-49ab-b07a-1a0d8893eded None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:06.443378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febfc60-896f-4e49-9bde-0e76e79ab1c9 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:06.465778 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:06.483139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d957eab-c398-42b8-8109-cdd3d32cf521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:06.485001 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d957eab-c398-42b8-8109-cdd3d32cf521 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:06.485632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d957eab-c398-42b8-8109-cdd3d32cf521 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:06.486203 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d957eab-c398-42b8-8109-cdd3d32cf521 None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:06.487745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d957eab-c398-42b8-8109-cdd3d32cf521 None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:06.489750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d957eab-c398-42b8-8109-cdd3d32cf521 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:06.491746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 60/124] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:06 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:06.492714 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d957eab-c398-42b8-8109-cdd3d32cf521 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:06.550134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324cc8e-e8d3-49ab-b07a-1a0d8893eded None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:06.588689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3fb01-da4d-4347-8ffc-524786866c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:06.589524 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3fb01-da4d-4347-8ffc-524786866c37 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:06.589815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3fb01-da4d-4347-8ffc-524786866c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:06.590098 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e3fb01-da4d-4347-8ffc-524786866c37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:06.600867 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e3fb01-da4d-4347-8ffc-524786866c37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:06.608641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-72e3fb01-da4d-4347-8ffc-524786866c37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:06.608641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:06.608641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:06.608641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:06.608641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:06.608641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:06.608641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:06.608641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:06.608641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:06.608641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:06.608641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:06.608641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:06.608641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:06.608641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:06.608641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:06.608641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:06.610088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:06.610088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:06.610088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:06.610088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:06.610088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:06.610088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:06.610088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:06.610088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:06.610088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:06.610088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:06.610088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:06.610088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:06.610088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:06.610088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:06.610088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:06.610088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:06.610910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:06.610910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:06.610910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:06.610910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:06.610910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:06.610910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:06.610910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:06.610910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:06.610910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:06.610910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:06.610910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:06.610910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:06.610910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 65/125] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:06.611534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3fb01-da4d-4347-8ffc-524786866c37 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:06.626531 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d957eab-c398-42b8-8109-cdd3d32cf521 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:06.665101 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:06.676571 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbca16-76fd-4cad-b03d-1a3eaeee9931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:06.677896 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbca16-76fd-4cad-b03d-1a3eaeee9931 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:06.678379 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbca16-76fd-4cad-b03d-1a3eaeee9931 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:06.678951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbca16-76fd-4cad-b03d-1a3eaeee9931 None admin] PATH_INFO: `/v3/users/d01511dd98c7416fb8387a8557165556` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:06.680646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbca16-76fd-4cad-b03d-1a3eaeee9931 None admin] RBAC: Authorizing `identity:get_user(user_id=d01511dd98c7416fb8387a8557165556)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:06.682289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbca16-76fd-4cad-b03d-1a3eaeee9931 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:06.684445 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 61/126] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:06 2022] GET /identity/v3/users/d01511dd98c7416fb8387a8557165556 => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:06.685479 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbca16-76fd-4cad-b03d-1a3eaeee9931 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:06.720239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e3fb01-da4d-4347-8ffc-524786866c37 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:06.749132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:06.755673 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8cb49-a799-41f3-975c-f0bfcb192b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:06.756363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8cb49-a799-41f3-975c-f0bfcb192b33 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:06.756641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8cb49-a799-41f3-975c-f0bfcb192b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:06.756930 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8cb49-a799-41f3-975c-f0bfcb192b33 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:06.757692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8cb49-a799-41f3-975c-f0bfcb192b33 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:06.758009 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8cb49-a799-41f3-975c-f0bfcb192b33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:06.759454 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8cb49-a799-41f3-975c-f0bfcb192b33 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:06.773201 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 66/127] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:06.774119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8cb49-a799-41f3-975c-f0bfcb192b33 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:06.806034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbca16-76fd-4cad-b03d-1a3eaeee9931 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:06.844963 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9a27c-5632-499a-a4a6-89ad40243e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:06.846043 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9a27c-5632-499a-a4a6-89ad40243e3e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:06.846524 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9a27c-5632-499a-a4a6-89ad40243e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:06.847083 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9a27c-5632-499a-a4a6-89ad40243e3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:06.848300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9a27c-5632-499a-a4a6-89ad40243e3e None admin] RBAC: Adding query filter params (role.id=559659e5dbc346d79735375b9130736d, scope.project.id=81fa154d1fc04266abee9f0051cba2fa, user.id=07188900e83d448685f5ee569bb0b6da) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:06.848809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9a27c-5632-499a-a4a6-89ad40243e3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:06.850474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9a27c-5632-499a-a4a6-89ad40243e3e None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:06.857087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 62/128] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:06 2022] GET /identity/v3/role_assignments?user.id=07188900e83d448685f5ee569bb0b6da&scope.project.id=81fa154d1fc04266abee9f0051cba2fa&role.id=559659e5dbc346d79735375b9130736d => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:06.858181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9a27c-5632-499a-a4a6-89ad40243e3e None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:06.896815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8cb49-a799-41f3-975c-f0bfcb192b33 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:06.939120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:06.979036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9a27c-5632-499a-a4a6-89ad40243e3e None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:07.003274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:07.007456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d2ca4-ca34-4889-b200-39940bd8bfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:07.008392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d2ca4-ca34-4889-b200-39940bd8bfa9 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:07.011031 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d2ca4-ca34-4889-b200-39940bd8bfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:07.011031 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d2ca4-ca34-4889-b200-39940bd8bfa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:07.011031 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d2ca4-ca34-4889-b200-39940bd8bfa9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:07.011031 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d2ca4-ca34-4889-b200-39940bd8bfa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:07.011342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d2ca4-ca34-4889-b200-39940bd8bfa9 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:07.021120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b19eac-384c-4c83-8453-05626274e75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:07.022004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b19eac-384c-4c83-8453-05626274e75c None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:07.022336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b19eac-384c-4c83-8453-05626274e75c None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:07.022621 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b19eac-384c-4c83-8453-05626274e75c None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:07.024043 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b19eac-384c-4c83-8453-05626274e75c None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:07.025506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b19eac-384c-4c83-8453-05626274e75c None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:07.027686 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 63/129] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:06 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:07.028558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b19eac-384c-4c83-8453-05626274e75c None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:07.030600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 67/130] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:07.038001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d2ca4-ca34-4889-b200-39940bd8bfa9 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:07.146779 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b19eac-384c-4c83-8453-05626274e75c None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:07.153512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d2ca4-ca34-4889-b200-39940bd8bfa9 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:07.188765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ebcbd-fe75-43ff-97ed-9b0e62dc28c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:07.189667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ebcbd-fe75-43ff-97ed-9b0e62dc28c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:07.189952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ebcbd-fe75-43ff-97ed-9b0e62dc28c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:07.190234 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ebcbd-fe75-43ff-97ed-9b0e62dc28c4 None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa/users/d01511dd98c7416fb8387a8557165556/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:07.192270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ebcbd-fe75-43ff-97ed-9b0e62dc28c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=81fa154d1fc04266abee9f0051cba2fa, user_id=d01511dd98c7416fb8387a8557165556, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:07.193722 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ebcbd-fe75-43ff-97ed-9b0e62dc28c4 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:07.199401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df70f3-3c70-49db-865f-8dc72d06e8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:07.200435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df70f3-3c70-49db-865f-8dc72d06e8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:07.200733 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df70f3-3c70-49db-865f-8dc72d06e8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:07.201067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df70f3-3c70-49db-865f-8dc72d06e8e8 None admin] PATH_INFO: `/v3/users/7e0b262e8410401e8d9297ca2732da45` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:07.207645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df70f3-3c70-49db-865f-8dc72d06e8e8 None admin] RBAC: Authorizing `identity:get_user(user_id=7e0b262e8410401e8d9297ca2732da45)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:07.209358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df70f3-3c70-49db-865f-8dc72d06e8e8 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:07.211509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 64/131] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:07 2022] GET /identity/v3/users/7e0b262e8410401e8d9297ca2732da45 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:07.212442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df70f3-3c70-49db-865f-8dc72d06e8e8 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:07.247408 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 68/132] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 08:59:07 2022] PUT /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa/users/d01511dd98c7416fb8387a8557165556/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:59:07.248142 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ebcbd-fe75-43ff-97ed-9b0e62dc28c4 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:07.319893 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:07.353076 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df70f3-3c70-49db-865f-8dc72d06e8e8 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:07.397724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:07.420409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f1b41-80ef-40ce-971a-3a757940dd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:07.422642 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f1b41-80ef-40ce-971a-3a757940dd36 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:07.423357 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f1b41-80ef-40ce-971a-3a757940dd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:07.423858 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f1b41-80ef-40ce-971a-3a757940dd36 None admin] PATH_INFO: `/v3/users/7e0b262e8410401e8d9297ca2732da45` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:07.427446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f1b41-80ef-40ce-971a-3a757940dd36 None admin] RBAC: Authorizing `identity:get_user(user_id=7e0b262e8410401e8d9297ca2732da45)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:07.429181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f1b41-80ef-40ce-971a-3a757940dd36 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:07.431133 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 65/133] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:07 2022] GET /identity/v3/users/7e0b262e8410401e8d9297ca2732da45 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:07.437159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f1b41-80ef-40ce-971a-3a757940dd36 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:07.576580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ebcbd-fe75-43ff-97ed-9b0e62dc28c4 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:07.581436 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f1b41-80ef-40ce-971a-3a757940dd36 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:07.613580 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:07.623980 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27271c97-f5cd-4cc4-9a0f-f30ffa9a3a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:07.625548 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27271c97-f5cd-4cc4-9a0f-f30ffa9a3a52 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:07.625968 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27271c97-f5cd-4cc4-9a0f-f30ffa9a3a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:07.626340 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27271c97-f5cd-4cc4-9a0f-f30ffa9a3a52 None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:07.628111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27271c97-f5cd-4cc4-9a0f-f30ffa9a3a52 None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:07.629587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27271c97-f5cd-4cc4-9a0f-f30ffa9a3a52 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:07.630478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54aba37-3d04-4077-a89a-d84553d7ee5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:07.631417 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 69/134] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:07 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:07.631842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aba37-3d04-4077-a89a-d84553d7ee5f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:07.632351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aba37-3d04-4077-a89a-d84553d7ee5f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:07.632906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54aba37-3d04-4077-a89a-d84553d7ee5f None admin] PATH_INFO: `/v3/projects/3366828c9f264c46a2d21ee94c2e705d` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:07.634331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54aba37-3d04-4077-a89a-d84553d7ee5f None admin] RBAC: Authorizing `identity:get_project(project_id=3366828c9f264c46a2d21ee94c2e705d)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:07.636341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54aba37-3d04-4077-a89a-d84553d7ee5f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:07.637070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27271c97-f5cd-4cc4-9a0f-f30ffa9a3a52 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:07.638670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 66/135] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:07 2022] GET /identity/v3/projects/3366828c9f264c46a2d21ee94c2e705d => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:07.644878 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54aba37-3d04-4077-a89a-d84553d7ee5f None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:07.746691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27271c97-f5cd-4cc4-9a0f-f30ffa9a3a52 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:07.748122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:07.795241 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f348d-cb93-4311-a816-df9a875efbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:07.796405 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f348d-cb93-4311-a816-df9a875efbeb None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:07.798049 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f348d-cb93-4311-a816-df9a875efbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:07.798475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f348d-cb93-4311-a816-df9a875efbeb None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa/users/7e0b262e8410401e8d9297ca2732da45/roles/05ca4843c34e463c94de09f756d95318` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:07.802521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f348d-cb93-4311-a816-df9a875efbeb None admin] RBAC: Authorizing `identity:create_grant(project_id=81fa154d1fc04266abee9f0051cba2fa, user_id=7e0b262e8410401e8d9297ca2732da45, role_id=05ca4843c34e463c94de09f756d95318)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:07.805285 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f348d-cb93-4311-a816-df9a875efbeb None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:07.811085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54aba37-3d04-4077-a89a-d84553d7ee5f None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:07.830096 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 70/136] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 08:59:07 2022] PUT /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa/users/7e0b262e8410401e8d9297ca2732da45/roles/05ca4843c34e463c94de09f756d95318 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:59:07.859813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f17e4f-d49b-4181-a049-061220b319a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:07.861057 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f17e4f-d49b-4181-a049-061220b319a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:07.861493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f17e4f-d49b-4181-a049-061220b319a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:07.861934 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f17e4f-d49b-4181-a049-061220b319a6 None admin] PATH_INFO: `/v3/projects/3366828c9f264c46a2d21ee94c2e705d/users/7e0b262e8410401e8d9297ca2732da45/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:07.864333 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f17e4f-d49b-4181-a049-061220b319a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=3366828c9f264c46a2d21ee94c2e705d, user_id=7e0b262e8410401e8d9297ca2732da45, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:07.866139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f17e4f-d49b-4181-a049-061220b319a6 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:07.882501 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 67/137] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 08:59:07 2022] PUT /identity/v3/projects/3366828c9f264c46a2d21ee94c2e705d/users/7e0b262e8410401e8d9297ca2732da45/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:59:08.127356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063d785-e0a0-4ac6-939b-f448e43a7bef None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:08.127932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063d785-e0a0-4ac6-939b-f448e43a7bef None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:08.128366 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063d785-e0a0-4ac6-939b-f448e43a7bef None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:08.130245 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 71/138] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:08.140662 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfc933-95e1-4863-a83c-0328286175eb None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:08.141195 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfc933-95e1-4863-a83c-0328286175eb None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:08.141636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cfc933-95e1-4863-a83c-0328286175eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:08.163778 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-e1cfc933-95e1-4863-a83c-0328286175eb None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:08.165258 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e1cfc933-95e1-4863-a83c-0328286175eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:08.221364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:08.243844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 68/139] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:08.293231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063d785-e0a0-4ac6-939b-f448e43a7bef None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:08.395980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b2beb-afc0-42fe-80af-be8ca2cce627 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:08.396269 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b2beb-afc0-42fe-80af-be8ca2cce627 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:08.396464 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b2beb-afc0-42fe-80af-be8ca2cce627 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:08.397848 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 69/140] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:59:08.403197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9de59-7054-48d5-bf04-914287ba31db None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:08.403485 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9de59-7054-48d5-bf04-914287ba31db None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:08.403769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba9de59-7054-48d5-bf04-914287ba31db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:08.414325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063d785-e0a0-4ac6-939b-f448e43a7bef None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:08.443743 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-2ba9de59-7054-48d5-bf04-914287ba31db None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:08.445279 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2ba9de59-7054-48d5-bf04-914287ba31db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:08.471179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c8c44-e6dc-4dc0-9104-f2fcea2ac101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:08.472066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c8c44-e6dc-4dc0-9104-f2fcea2ac101 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:08.472363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c8c44-e6dc-4dc0-9104-f2fcea2ac101 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:08.472640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c8c44-e6dc-4dc0-9104-f2fcea2ac101 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:08.495704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c8c44-e6dc-4dc0-9104-f2fcea2ac101 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:08.504372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-115c8c44-e6dc-4dc0-9104-f2fcea2ac101 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:08.504372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:08.504372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:08.504372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:08.504372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:08.504372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:08.504372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:08.504372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:08.504372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:08.504372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:08.504372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:08.504372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:08.504372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:08.504372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:08.504372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:08.504372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:08.505272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:08.505272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:08.505272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:08.505272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:08.505272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:08.505272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:08.505272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:08.505272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:08.505272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:08.505272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:08.505272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:08.505272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:08.505272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:08.505272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:08.505272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:08.505272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:08.506091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:08.506091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:08.506091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:08.506091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:08.506091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:08.506091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:08.506091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:08.506091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:08.506091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:08.506091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:08.506091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:08.506091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:08.506091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 72/141] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:08.511480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c8c44-e6dc-4dc0-9104-f2fcea2ac101 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:08.567067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:08.585043 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 70/142] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:08.633975 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9de59-7054-48d5-bf04-914287ba31db None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:08.676413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c8c44-e6dc-4dc0-9104-f2fcea2ac101 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:08.760354 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fdee3-93f7-4fc3-8a1d-3e291dfca4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:08.761442 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fdee3-93f7-4fc3-8a1d-3e291dfca4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:08.761728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fdee3-93f7-4fc3-8a1d-3e291dfca4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:08.762140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fdee3-93f7-4fc3-8a1d-3e291dfca4d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:08.762933 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fdee3-93f7-4fc3-8a1d-3e291dfca4d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:08.763240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fdee3-93f7-4fc3-8a1d-3e291dfca4d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:08.764681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fdee3-93f7-4fc3-8a1d-3e291dfca4d4 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:08.783314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 73/143] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:08.794833 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fdee3-93f7-4fc3-8a1d-3e291dfca4d4 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:08.846322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba9de59-7054-48d5-bf04-914287ba31db None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:08.873084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:08.927488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4d8c5-cabe-4c21-837a-40d7a022d828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:08.928238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4d8c5-cabe-4c21-837a-40d7a022d828 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:08.928530 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4d8c5-cabe-4c21-837a-40d7a022d828 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:08.928813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4d8c5-cabe-4c21-837a-40d7a022d828 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:08.930157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4d8c5-cabe-4c21-837a-40d7a022d828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:08.931677 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4d8c5-cabe-4c21-837a-40d7a022d828 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:08.933765 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 71/144] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:08.939574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4d8c5-cabe-4c21-837a-40d7a022d828 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:08.953441 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:08.978863 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fdee3-93f7-4fc3-8a1d-3e291dfca4d4 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:09.031476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c5e52-187e-4d79-8105-882b45712bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:09.032362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c5e52-187e-4d79-8105-882b45712bfd None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:09.032695 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c5e52-187e-4d79-8105-882b45712bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:09.032973 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c5e52-187e-4d79-8105-882b45712bfd None admin] PATH_INFO: `/v3/users/7e0b262e8410401e8d9297ca2732da45` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:09.034245 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c5e52-187e-4d79-8105-882b45712bfd None admin] RBAC: Authorizing `identity:get_user(user_id=7e0b262e8410401e8d9297ca2732da45)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:09.036687 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c5e52-187e-4d79-8105-882b45712bfd None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:09.038338 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 74/145] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:08 2022] GET /identity/v3/users/7e0b262e8410401e8d9297ca2732da45 => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:09.044039 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c5e52-187e-4d79-8105-882b45712bfd None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:09.190071 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4d8c5-cabe-4c21-837a-40d7a022d828 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:09.273817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07180fc4-4d52-41d2-b50d-c9331060ba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:09.274589 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07180fc4-4d52-41d2-b50d-c9331060ba99 None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:09.274832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07180fc4-4d52-41d2-b50d-c9331060ba99 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:09.275168 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07180fc4-4d52-41d2-b50d-c9331060ba99 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:09.275927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07180fc4-4d52-41d2-b50d-c9331060ba99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:09.277386 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07180fc4-4d52-41d2-b50d-c9331060ba99 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:09.284263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c5e52-187e-4d79-8105-882b45712bfd None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:09.349111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 72/146] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 08:59:08 2022] POST /identity/v3/users => generated 283 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:59:09.356925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a91453-7dc2-413b-9de7-838a8bbe0d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:09.357988 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a91453-7dc2-413b-9de7-838a8bbe0d1a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:09.358494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a91453-7dc2-413b-9de7-838a8bbe0d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:09.359692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a91453-7dc2-413b-9de7-838a8bbe0d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:09.399414 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:09.481997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a91453-7dc2-413b-9de7-838a8bbe0d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:09.485709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a91453-7dc2-413b-9de7-838a8bbe0d1a None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:09.636081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 75/147] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:09.646517 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07180fc4-4d52-41d2-b50d-c9331060ba99 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:09.738702 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:09.783176 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bcd9c-b0d6-49ba-8956-54b31389bf64 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:09.783913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bcd9c-b0d6-49ba-8956-54b31389bf64 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:09.784372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bcd9c-b0d6-49ba-8956-54b31389bf64 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:09.786835 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 76/148] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:09.792308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5822b7-1190-4794-a197-f92a50e3cab2 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:09.792931 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5822b7-1190-4794-a197-f92a50e3cab2 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:09.793289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5822b7-1190-4794-a197-f92a50e3cab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:09.821157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-2a5822b7-1190-4794-a197-f92a50e3cab2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:09.822496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-2a5822b7-1190-4794-a197-f92a50e3cab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:09.857279 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07180fc4-4d52-41d2-b50d-c9331060ba99 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:09.911672 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:09.931524 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105f64f-180b-4d22-b07b-3d58115164cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:09.932798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105f64f-180b-4d22-b07b-3d58115164cc None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:09.933267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105f64f-180b-4d22-b07b-3d58115164cc None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:09.933970 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105f64f-180b-4d22-b07b-3d58115164cc None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:09.935512 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105f64f-180b-4d22-b07b-3d58115164cc None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:09.937725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105f64f-180b-4d22-b07b-3d58115164cc None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:09.944859 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 73/149] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:09 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:09.946417 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105f64f-180b-4d22-b07b-3d58115164cc None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:09.955809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 77/150] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:10.024141 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5822b7-1190-4794-a197-f92a50e3cab2 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:10.116971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105f64f-180b-4d22-b07b-3d58115164cc None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:10.123785 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:10.158818 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5822b7-1190-4794-a197-f92a50e3cab2 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:10.178479 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4966a-1b28-49f4-9257-4afca56b448c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:10.179176 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4966a-1b28-49f4-9257-4afca56b448c None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:10.179467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4966a-1b28-49f4-9257-4afca56b448c None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:10.179748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4966a-1b28-49f4-9257-4afca56b448c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:10.180598 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4966a-1b28-49f4-9257-4afca56b448c None admin] RBAC: Adding query filter params (role.id=0e2011aac3e1430cb96bdbf810e89eb7, scope.project.id=81fa154d1fc04266abee9f0051cba2fa, user.id=7e0b262e8410401e8d9297ca2732da45) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:10.180916 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4966a-1b28-49f4-9257-4afca56b448c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:10.182391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4966a-1b28-49f4-9257-4afca56b448c None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:10.197008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0689b-14f0-4935-9c7a-1bafe03f03b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:10.197430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 74/151] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:09 2022] GET /identity/v3/role_assignments?user.id=7e0b262e8410401e8d9297ca2732da45&scope.project.id=81fa154d1fc04266abee9f0051cba2fa&role.id=0e2011aac3e1430cb96bdbf810e89eb7 => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:10.198100 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0689b-14f0-4935-9c7a-1bafe03f03b7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:10.198384 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0689b-14f0-4935-9c7a-1bafe03f03b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:10.198660 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0689b-14f0-4935-9c7a-1bafe03f03b7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:10.211411 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce0689b-14f0-4935-9c7a-1bafe03f03b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:10.220413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-8ce0689b-14f0-4935-9c7a-1bafe03f03b7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:10.220413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:10.220413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:10.220413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:10.220413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:10.220413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:10.220413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:10.220413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:10.220413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:10.220413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:10.220413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:10.220413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:10.220413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:10.220413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:10.220413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:10.220413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:10.221361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:10.221361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:10.221361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:10.221361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:10.221361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:10.221361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:10.221361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:10.221361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:10.221361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:10.221361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:10.221361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:10.221361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:10.221361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:10.221361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:10.221361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:10.221361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:10.222580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:10.222580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:10.222580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:10.222580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:10.222580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:10.222580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:10.222580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:10.222580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:10.222580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:10.222580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:10.222580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:10.222580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:10.222580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 78/152] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:10.229226 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4966a-1b28-49f4-9257-4afca56b448c None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:10.373075 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4966a-1b28-49f4-9257-4afca56b448c None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:10.419723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c705d2-93f7-41e8-8663-84cb23e9382f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:10.420411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c705d2-93f7-41e8-8663-84cb23e9382f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:10.420700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c705d2-93f7-41e8-8663-84cb23e9382f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:10.420985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c705d2-93f7-41e8-8663-84cb23e9382f None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:10.422403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c705d2-93f7-41e8-8663-84cb23e9382f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:10.422739 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c705d2-93f7-41e8-8663-84cb23e9382f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:10.424217 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c705d2-93f7-41e8-8663-84cb23e9382f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:10.436443 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 75/153] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:10.442280 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0689b-14f0-4935-9c7a-1bafe03f03b7 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:10.506893 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:10.588037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0689b-14f0-4935-9c7a-1bafe03f03b7 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:10.605386 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40354f-0e20-4b65-8944-c4164714dc09 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:10.606011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40354f-0e20-4b65-8944-c4164714dc09 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:10.606500 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40354f-0e20-4b65-8944-c4164714dc09 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:10.608353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 76/154] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:10.614597 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f73b6-8bbd-4ac1-9e1b-08f60af18b04 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:10.615211 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f73b6-8bbd-4ac1-9e1b-08f60af18b04 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:10.615755 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f73b6-8bbd-4ac1-9e1b-08f60af18b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:10.640143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17c5f4-9709-4643-9967-6cf8369f2fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:10.640982 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17c5f4-9709-4643-9967-6cf8369f2fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:10.641333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17c5f4-9709-4643-9967-6cf8369f2fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:10.641625 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17c5f4-9709-4643-9967-6cf8369f2fc4 None admin] PATH_INFO: `/v3/users/07188900e83d448685f5ee569bb0b6da` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:10.643373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17c5f4-9709-4643-9967-6cf8369f2fc4 None admin] RBAC: Authorizing `identity:get_user(user_id=07188900e83d448685f5ee569bb0b6da)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:10.644851 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17c5f4-9709-4643-9967-6cf8369f2fc4 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:10.646487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 79/155] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:10 2022] GET /identity/v3/users/07188900e83d448685f5ee569bb0b6da => generated 289 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:10.652050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17c5f4-9709-4643-9967-6cf8369f2fc4 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:10.659724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-594f73b6-8bbd-4ac1-9e1b-08f60af18b04 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:10.660939 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-594f73b6-8bbd-4ac1-9e1b-08f60af18b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:10.707044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:10.776759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 77/156] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:10.823785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17c5f4-9709-4643-9967-6cf8369f2fc4 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:10.828657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f73b6-8bbd-4ac1-9e1b-08f60af18b04 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:10.837510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:10.868966 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c22efa-9976-4a70-92e6-502ea41781d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:10.870092 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c22efa-9976-4a70-92e6-502ea41781d7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:10.870501 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c22efa-9976-4a70-92e6-502ea41781d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:10.870935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c22efa-9976-4a70-92e6-502ea41781d7 None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:10.874512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c22efa-9976-4a70-92e6-502ea41781d7 None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:10.876431 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c22efa-9976-4a70-92e6-502ea41781d7 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:10.879460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 80/157] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:10 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:10.889481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c22efa-9976-4a70-92e6-502ea41781d7 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:11.046980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f73b6-8bbd-4ac1-9e1b-08f60af18b04 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:11.122600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c22efa-9976-4a70-92e6-502ea41781d7 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:11.136760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e263a-f331-4443-8ded-2f8c62b09722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:11.137573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e263a-f331-4443-8ded-2f8c62b09722 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:11.137866 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e263a-f331-4443-8ded-2f8c62b09722 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:11.138123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e263a-f331-4443-8ded-2f8c62b09722 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:11.169625 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e263a-f331-4443-8ded-2f8c62b09722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:11.177804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-2a9e263a-f331-4443-8ded-2f8c62b09722 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:11.177804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:11.177804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:11.177804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:11.177804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:11.177804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:11.177804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:11.177804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:11.177804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:11.177804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:11.177804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:11.177804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:11.177804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:11.177804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:11.177804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:11.177804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:11.178847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:11.178847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:11.178847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:11.178847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:11.178847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:11.178847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:11.178847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:11.178847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:11.178847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:11.178847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:11.178847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:11.178847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:11.178847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:11.178847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:11.178847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:11.178847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:11.179631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:11.179631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:11.179631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:11.179631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:11.179631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:11.179631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:11.179631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:11.179631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:11.179631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:11.179631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:11.179631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:11.179631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:11.183144 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 78/158] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 08:59:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:11.190250 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e263a-f331-4443-8ded-2f8c62b09722 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:11.205166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25b28e-c309-4201-9534-9b122ec6d68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:11.206078 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25b28e-c309-4201-9534-9b122ec6d68b None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:11.206362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25b28e-c309-4201-9534-9b122ec6d68b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:11.206643 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25b28e-c309-4201-9534-9b122ec6d68b None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa/users/07188900e83d448685f5ee569bb0b6da/roles/559659e5dbc346d79735375b9130736d` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:11.208739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25b28e-c309-4201-9534-9b122ec6d68b None admin] RBAC: Authorizing `identity:create_grant(project_id=81fa154d1fc04266abee9f0051cba2fa, user_id=07188900e83d448685f5ee569bb0b6da, role_id=559659e5dbc346d79735375b9130736d)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:11.210216 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25b28e-c309-4201-9534-9b122ec6d68b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:11.240657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:11.245707 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 81/159] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 08:59:10 2022] PUT /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa/users/07188900e83d448685f5ee569bb0b6da/roles/559659e5dbc346d79735375b9130736d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:59:11.354133 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e263a-f331-4443-8ded-2f8c62b09722 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:11.397627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1e069-a0c2-458f-be0a-dd9788d1af85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:11.398403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1e069-a0c2-458f-be0a-dd9788d1af85 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:11.398579 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1e069-a0c2-458f-be0a-dd9788d1af85 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:11.398882 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1e069-a0c2-458f-be0a-dd9788d1af85 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:11.399639 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1e069-a0c2-458f-be0a-dd9788d1af85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:11.399946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1e069-a0c2-458f-be0a-dd9788d1af85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:11.401328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1e069-a0c2-458f-be0a-dd9788d1af85 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:11.413830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 79/160] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:11.419967 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25b28e-c309-4201-9534-9b122ec6d68b None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:11.489174 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:11.515556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d8d07-15a2-4c33-8894-a6ff09f99581 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:11.516032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d8d07-15a2-4c33-8894-a6ff09f99581 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:11.516312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d8d07-15a2-4c33-8894-a6ff09f99581 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:11.517710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 80/161] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:11.528077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e11fc-2c00-4da8-a8f8-3fd981002c95 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:11.528367 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e11fc-2c00-4da8-a8f8-3fd981002c95 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:11.528635 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e11fc-2c00-4da8-a8f8-3fd981002c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:11.572199 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-5e9e11fc-2c00-4da8-a8f8-3fd981002c95 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:11.573573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5e9e11fc-2c00-4da8-a8f8-3fd981002c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:11.581140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25b28e-c309-4201-9534-9b122ec6d68b None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:11.653705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3531152-a52f-4153-b753-e4add8f75620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:11.654511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3531152-a52f-4153-b753-e4add8f75620 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:11.654822 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3531152-a52f-4153-b753-e4add8f75620 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:11.655102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3531152-a52f-4153-b753-e4add8f75620 None admin] PATH_INFO: `/v3/users/d01511dd98c7416fb8387a8557165556` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:11.656763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3531152-a52f-4153-b753-e4add8f75620 None admin] RBAC: Authorizing `identity:get_user(user_id=d01511dd98c7416fb8387a8557165556)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:11.658228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3531152-a52f-4153-b753-e4add8f75620 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:11.662226 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 82/162] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:11 2022] GET /identity/v3/users/d01511dd98c7416fb8387a8557165556 => generated 247 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:11.663483 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09613e82-d4a1-4e3b-8d07-34b8ecbcbb1c None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:11.663770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09613e82-d4a1-4e3b-8d07-34b8ecbcbb1c None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:11.664050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09613e82-d4a1-4e3b-8d07-34b8ecbcbb1c None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:11.665307 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 83/163] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:11.672304 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09613e82-d4a1-4e3b-8d07-34b8ecbcbb1c None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:11.711844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 81/164] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:11.713161 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5424-5771-473f-957d-bdb8d2c6613d None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:11.713457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5424-5771-473f-957d-bdb8d2c6613d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:11.713740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5424-5771-473f-957d-bdb8d2c6613d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:11.757732 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-d49e5424-5771-473f-957d-bdb8d2c6613d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:11.758976 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d49e5424-5771-473f-957d-bdb8d2c6613d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:11.774270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:11.864025 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09613e82-d4a1-4e3b-8d07-34b8ecbcbb1c None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:11.879587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:11.892790 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 82/165] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:11.893689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e5424-5771-473f-957d-bdb8d2c6613d None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:11.913188 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064a970-0c51-40d5-a61b-7b763161c7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:11.913774 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064a970-0c51-40d5-a61b-7b763161c7bf None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:11.914062 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064a970-0c51-40d5-a61b-7b763161c7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:11.914347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064a970-0c51-40d5-a61b-7b763161c7bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:12.033063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064a970-0c51-40d5-a61b-7b763161c7bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:12.034558 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064a970-0c51-40d5-a61b-7b763161c7bf None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:12.038720 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e5424-5771-473f-957d-bdb8d2c6613d None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:12.114783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163bd15-c810-4a23-9d4a-5fc4213a2b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:12.115606 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163bd15-c810-4a23-9d4a-5fc4213a2b62 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:12.115888 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163bd15-c810-4a23-9d4a-5fc4213a2b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:12.116164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163bd15-c810-4a23-9d4a-5fc4213a2b62 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:12.140477 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163bd15-c810-4a23-9d4a-5fc4213a2b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:12.148581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-8163bd15-c810-4a23-9d4a-5fc4213a2b62 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:12.148581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:12.148581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:12.148581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:12.148581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:12.148581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:12.148581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:12.148581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:12.148581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:12.148581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:12.148581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:12.148581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:12.148581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:12.148581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:12.148581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:12.148581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:12.149430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:12.149430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:12.149430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:12.149430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:12.149430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:12.149430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:12.149430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:12.149430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:12.149430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:12.149430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:12.149430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:12.149430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:12.149430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:12.149430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:12.149430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:12.149430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:12.150476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:12.150476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:12.150476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:12.150476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:12.150476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:12.150476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:12.150476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:12.150476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:12.150476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:12.150476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:12.150476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:12.150476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:12.150476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 83/166] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 08:59:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:12.151151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163bd15-c810-4a23-9d4a-5fc4213a2b62 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:12.163558 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 84/167] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:12.164340 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064a970-0c51-40d5-a61b-7b763161c7bf None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:12.177515 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:12.180581 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:12.274524 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064a970-0c51-40d5-a61b-7b763161c7bf None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:12.310083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3b52f-cf9d-4b46-8e55-589e1627a105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:12.310808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3b52f-cf9d-4b46-8e55-589e1627a105 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:12.311100 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3b52f-cf9d-4b46-8e55-589e1627a105 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:12.311384 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3b52f-cf9d-4b46-8e55-589e1627a105 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:12.312225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3b52f-cf9d-4b46-8e55-589e1627a105 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:12.312542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3b52f-cf9d-4b46-8e55-589e1627a105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:12.313938 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3b52f-cf9d-4b46-8e55-589e1627a105 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:12.331535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 85/168] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:12.332164 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3b52f-cf9d-4b46-8e55-589e1627a105 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:12.411079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:12.444224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163bd15-c810-4a23-9d4a-5fc4213a2b62 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:12.451522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3b52f-cf9d-4b46-8e55-589e1627a105 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:12.499670 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd012393-88e3-4b98-97de-cff92ac527fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:12.500351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd012393-88e3-4b98-97de-cff92ac527fa None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:12.500637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd012393-88e3-4b98-97de-cff92ac527fa None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:12.500921 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd012393-88e3-4b98-97de-cff92ac527fa None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:12.502347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd012393-88e3-4b98-97de-cff92ac527fa None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:12.503846 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd012393-88e3-4b98-97de-cff92ac527fa None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:12.506078 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 86/169] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:12 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:12.511178 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd012393-88e3-4b98-97de-cff92ac527fa None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:12.511929 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667684d-44d0-4b8e-8908-3d960683f4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:12.512733 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667684d-44d0-4b8e-8908-3d960683f4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:12.513013 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667684d-44d0-4b8e-8908-3d960683f4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:12.513284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5667684d-44d0-4b8e-8908-3d960683f4b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:12.556340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5667684d-44d0-4b8e-8908-3d960683f4b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:12.576723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-5667684d-44d0-4b8e-8908-3d960683f4b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:12.576723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:12.576723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:12.576723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:12.576723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:12.576723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:12.576723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:12.576723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:12.576723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:12.576723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:12.576723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:12.576723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:12.576723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:12.576723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:12.576723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:12.576723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:12.578346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:12.578346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:12.578346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:12.578346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:12.578346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:12.578346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:12.578346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:12.578346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:12.578346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:12.578346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:12.578346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:12.578346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:12.578346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:12.578346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:12.578346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:12.578346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:12.579742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:12.579742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:12.579742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:12.579742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:12.579742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:12.579742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:12.579742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:12.579742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:12.579742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:12.579742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:12.579742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:12.579742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:12.579742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 84/170] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:12.587519 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667684d-44d0-4b8e-8908-3d960683f4b6 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:12.708059 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd012393-88e3-4b98-97de-cff92ac527fa None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:12.733429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:12.781256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81c788-ba08-4d25-b1dc-67d463331043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:12.782135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81c788-ba08-4d25-b1dc-67d463331043 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:12.782415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81c788-ba08-4d25-b1dc-67d463331043 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:12.782701 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81c788-ba08-4d25-b1dc-67d463331043 None admin] PATH_INFO: `/v3/users/7e0b262e8410401e8d9297ca2732da45` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:12.784960 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81c788-ba08-4d25-b1dc-67d463331043 None admin] RBAC: Authorizing `identity:get_user(user_id=7e0b262e8410401e8d9297ca2732da45)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:12.786413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81c788-ba08-4d25-b1dc-67d463331043 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:12.790943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 87/171] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:12 2022] GET /identity/v3/users/7e0b262e8410401e8d9297ca2732da45 => generated 275 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:12.791744 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81c788-ba08-4d25-b1dc-67d463331043 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:12.807000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:12.808863 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5667684d-44d0-4b8e-8908-3d960683f4b6 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:12.864297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01492a91-4a52-4de5-8677-315fa5d0b536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:12.864967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01492a91-4a52-4de5-8677-315fa5d0b536 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:12.865253 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01492a91-4a52-4de5-8677-315fa5d0b536 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:12.865532 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01492a91-4a52-4de5-8677-315fa5d0b536 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:12.866386 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01492a91-4a52-4de5-8677-315fa5d0b536 None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=81fa154d1fc04266abee9f0051cba2fa, user.id=d01511dd98c7416fb8387a8557165556) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:12.866703 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01492a91-4a52-4de5-8677-315fa5d0b536 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:12.868224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01492a91-4a52-4de5-8677-315fa5d0b536 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:12.882824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 85/172] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:12 2022] GET /identity/v3/role_assignments?user.id=d01511dd98c7416fb8387a8557165556&scope.project.id=81fa154d1fc04266abee9f0051cba2fa&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:12.883496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01492a91-4a52-4de5-8677-315fa5d0b536 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:13.059065 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01492a91-4a52-4de5-8677-315fa5d0b536 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:13.065174 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81c788-ba08-4d25-b1dc-67d463331043 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:13.103090 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:13.116427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba66546-ac37-4264-813d-29cdbf9bbbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:13.117101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba66546-ac37-4264-813d-29cdbf9bbbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:13.117382 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba66546-ac37-4264-813d-29cdbf9bbbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:13.117660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba66546-ac37-4264-813d-29cdbf9bbbf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:13.160883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bceec5-f62e-414b-bdaa-291a69878b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:13.161578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bceec5-f62e-414b-bdaa-291a69878b42 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:13.161869 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bceec5-f62e-414b-bdaa-291a69878b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:13.162150 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bceec5-f62e-414b-bdaa-291a69878b42 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:13.162952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bceec5-f62e-414b-bdaa-291a69878b42 None admin] RBAC: Adding query filter params (name=member) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:13.163262 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bceec5-f62e-414b-bdaa-291a69878b42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:13.164686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bceec5-f62e-414b-bdaa-291a69878b42 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:13.183231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 88/173] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:13.189849 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bceec5-f62e-414b-bdaa-291a69878b42 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:13.224646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba66546-ac37-4264-813d-29cdbf9bbbf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:13.226126 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba66546-ac37-4264-813d-29cdbf9bbbf3 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:13.323134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:13.360805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bceec5-f62e-414b-bdaa-291a69878b42 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:13.386873 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 86/174] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:13.388434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfa1ba-6b4f-450d-a07b-05996fed3a38 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:13.388723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfa1ba-6b4f-450d-a07b-05996fed3a38 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:13.389544 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfa1ba-6b4f-450d-a07b-05996fed3a38 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:13.395877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 87/175] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:13.397328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfa1ba-6b4f-450d-a07b-05996fed3a38 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:13.415066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d0c2d-14c4-4f08-8a8c-77555fc3ec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:13.416637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d0c2d-14c4-4f08-8a8c-77555fc3ec64 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:13.416930 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d0c2d-14c4-4f08-8a8c-77555fc3ec64 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:13.417211 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d0c2d-14c4-4f08-8a8c-77555fc3ec64 None admin] PATH_INFO: `/v3/users/7e0b262e8410401e8d9297ca2732da45` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:13.418754 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d0c2d-14c4-4f08-8a8c-77555fc3ec64 None admin] RBAC: Authorizing `identity:get_user(user_id=7e0b262e8410401e8d9297ca2732da45)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:13.420825 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d0c2d-14c4-4f08-8a8c-77555fc3ec64 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:13.423630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 89/176] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:13 2022] GET /identity/v3/users/7e0b262e8410401e8d9297ca2732da45 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:13.424828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47403e0-7cd1-4857-9059-2957703de090 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:13.425113 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47403e0-7cd1-4857-9059-2957703de090 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:13.425391 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47403e0-7cd1-4857-9059-2957703de090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:13.435911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:13.449885 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-b47403e0-7cd1-4857-9059-2957703de090 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:13.451381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b47403e0-7cd1-4857-9059-2957703de090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:13.540695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfa1ba-6b4f-450d-a07b-05996fed3a38 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:13.555637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 90/177] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:13.556735 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47403e0-7cd1-4857-9059-2957703de090 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:13.599299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1aa93-13f6-4416-92ca-124ec2fd31c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:13.600473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1aa93-13f6-4416-92ca-124ec2fd31c1 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:13.600766 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1aa93-13f6-4416-92ca-124ec2fd31c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:13.601044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1aa93-13f6-4416-92ca-124ec2fd31c1 None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:13.602697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1aa93-13f6-4416-92ca-124ec2fd31c1 None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:13.604441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1aa93-13f6-4416-92ca-124ec2fd31c1 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:13.606926 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 88/178] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:13 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:13.608192 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1aa93-13f6-4416-92ca-124ec2fd31c1 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:13.652109 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:13.697481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47403e0-7cd1-4857-9059-2957703de090 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:13.717625 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:13.725052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1aa93-13f6-4416-92ca-124ec2fd31c1 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:13.748159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681477b-52f9-439b-9cf3-0fcfd8eb16f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:13.750088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681477b-52f9-439b-9cf3-0fcfd8eb16f3 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:13.750657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681477b-52f9-439b-9cf3-0fcfd8eb16f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:13.751194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681477b-52f9-439b-9cf3-0fcfd8eb16f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:13.766725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400150d7-addc-4c12-baa9-13a5dba06f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:13.766725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400150d7-addc-4c12-baa9-13a5dba06f79 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:13.766725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400150d7-addc-4c12-baa9-13a5dba06f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:13.766725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400150d7-addc-4c12-baa9-13a5dba06f79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:13.772351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400150d7-addc-4c12-baa9-13a5dba06f79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:13.779840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-400150d7-addc-4c12-baa9-13a5dba06f79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:13.779840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:13.779840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:13.779840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:13.779840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:13.779840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:13.779840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:13.779840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:13.779840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:13.779840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:13.779840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:13.779840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:13.779840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:13.779840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:13.779840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:13.779840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:13.780849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:13.780849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:13.780849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:13.780849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:13.780849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:13.780849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:13.780849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:13.780849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:13.780849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:13.780849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:13.780849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:13.780849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:13.780849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:13.780849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:13.780849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:13.780849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:13.787753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:13.787753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:13.787753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:13.787753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:13.787753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:13.787753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:13.787753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:13.787753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:13.787753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:13.787753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:13.787753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:13.787753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:13.787753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 89/179] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 08:59:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:13.787753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400150d7-addc-4c12-baa9-13a5dba06f79 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:13.875649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681477b-52f9-439b-9cf3-0fcfd8eb16f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:13.885616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681477b-52f9-439b-9cf3-0fcfd8eb16f3 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:13.886563 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400150d7-addc-4c12-baa9-13a5dba06f79 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:13.954621 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f7301-e8c6-447f-8866-d5741c4c2eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:13.955302 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f7301-e8c6-447f-8866-d5741c4c2eaa None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:13.955585 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f7301-e8c6-447f-8866-d5741c4c2eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:13.955861 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f7301-e8c6-447f-8866-d5741c4c2eaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:13.956702 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f7301-e8c6-447f-8866-d5741c4c2eaa None admin] RBAC: Adding query filter params (role.id=05ca4843c34e463c94de09f756d95318, scope.project.id=81fa154d1fc04266abee9f0051cba2fa, user.id=7e0b262e8410401e8d9297ca2732da45) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:13.957014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f7301-e8c6-447f-8866-d5741c4c2eaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:13.958449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f7301-e8c6-447f-8866-d5741c4c2eaa None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:13.972214 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 90/180] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:13 2022] GET /identity/v3/role_assignments?user.id=7e0b262e8410401e8d9297ca2732da45&scope.project.id=81fa154d1fc04266abee9f0051cba2fa&role.id=05ca4843c34e463c94de09f756d95318 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:13.974111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f7301-e8c6-447f-8866-d5741c4c2eaa None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:14.011970 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:14.031600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:14.108513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 91/181] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:14.114173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b084596-094d-47c4-8e95-e7618ffe0f3a None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:14.114468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b084596-094d-47c4-8e95-e7618ffe0f3a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:14.114786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b084596-094d-47c4-8e95-e7618ffe0f3a None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:14.116340 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 92/182] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:14.116930 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b084596-094d-47c4-8e95-e7618ffe0f3a None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:14.154576 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f7301-e8c6-447f-8866-d5741c4c2eaa None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:14.195787 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3bd36-696e-4588-9ce9-7159959296f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:14.196879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3bd36-696e-4588-9ce9-7159959296f9 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:14.197385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3bd36-696e-4588-9ce9-7159959296f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:14.198347 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3bd36-696e-4588-9ce9-7159959296f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:14.199337 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3bd36-696e-4588-9ce9-7159959296f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:14.199960 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3bd36-696e-4588-9ce9-7159959296f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:14.201341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3bd36-696e-4588-9ce9-7159959296f9 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:14.211009 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 91/183] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:14.216095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bf561-26bc-442d-808d-54849a925e89 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:14.216382 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bf561-26bc-442d-808d-54849a925e89 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:14.216660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24bf561-26bc-442d-808d-54849a925e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:14.236059 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-b24bf561-26bc-442d-808d-54849a925e89 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:14.237244 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b24bf561-26bc-442d-808d-54849a925e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:14.277111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b084596-094d-47c4-8e95-e7618ffe0f3a None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:14.317303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e51ec-6b55-4d94-951a-8177b9c92488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:14.318166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e51ec-6b55-4d94-951a-8177b9c92488 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:14.318446 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e51ec-6b55-4d94-951a-8177b9c92488 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:14.318750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e51ec-6b55-4d94-951a-8177b9c92488 None admin] PATH_INFO: `/v3/projects/3366828c9f264c46a2d21ee94c2e705d` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:14.320165 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e51ec-6b55-4d94-951a-8177b9c92488 None admin] RBAC: Authorizing `identity:get_project(project_id=3366828c9f264c46a2d21ee94c2e705d)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:14.321606 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e51ec-6b55-4d94-951a-8177b9c92488 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:14.329193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 92/184] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:14.329350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 93/185] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:14 2022] GET /identity/v3/projects/3366828c9f264c46a2d21ee94c2e705d => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:14.329456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b95b31-37e7-4140-8bdb-67a0580507cf None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:14.329541 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e51ec-6b55-4d94-951a-8177b9c92488 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:14.329620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b95b31-37e7-4140-8bdb-67a0580507cf None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:14.329620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b95b31-37e7-4140-8bdb-67a0580507cf None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:14.329620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 93/186] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:14.329620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb21062-8640-41df-9195-39a2c6f6b18f None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:14.329620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb21062-8640-41df-9195-39a2c6f6b18f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:14.329620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb21062-8640-41df-9195-39a2c6f6b18f None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:14.333900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 94/187] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:14.337283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28d678-00c0-4157-907c-b56863d22ee3 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:14.337283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28d678-00c0-4157-907c-b56863d22ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:14.337283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28d678-00c0-4157-907c-b56863d22ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:14.351957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-2c28d678-00c0-4157-907c-b56863d22ee3 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:14.353135 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2c28d678-00c0-4157-907c-b56863d22ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:14.355931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:14.364495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:14.430745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 95/188] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:14.430745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddf81e-f89b-4376-b51e-b6f32fe43434 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:14.430745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddf81e-f89b-4376-b51e-b6f32fe43434 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:14.430745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddf81e-f89b-4376-b51e-b6f32fe43434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:14.435730 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e51ec-6b55-4d94-951a-8177b9c92488 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:14.437872 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-82ddf81e-f89b-4376-b51e-b6f32fe43434 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:14.439106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-82ddf81e-f89b-4376-b51e-b6f32fe43434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:14.472006 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e68f96-672c-4805-a0b3-9c1274b0afe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:14.472917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e68f96-672c-4805-a0b3-9c1274b0afe3 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:14.473199 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e68f96-672c-4805-a0b3-9c1274b0afe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:14.473474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e68f96-672c-4805-a0b3-9c1274b0afe3 None admin] PATH_INFO: `/v3/users/79c76db5606f4aa7a3a8c6bd45dfebf0` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:14.486264 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e68f96-672c-4805-a0b3-9c1274b0afe3 None admin] RBAC: Authorizing `identity:get_user(user_id=79c76db5606f4aa7a3a8c6bd45dfebf0)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:14.487759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e68f96-672c-4805-a0b3-9c1274b0afe3 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:14.489435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 94/189] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:14 2022] GET /identity/v3/users/79c76db5606f4aa7a3a8c6bd45dfebf0 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:14.491502 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e68f96-672c-4805-a0b3-9c1274b0afe3 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:14.508746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 96/190] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:14.514861 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddf81e-f89b-4376-b51e-b6f32fe43434 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:14.582870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:14.587986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:14.592785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e68f96-672c-4805-a0b3-9c1274b0afe3 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:14.619770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddf81e-f89b-4376-b51e-b6f32fe43434 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:14.628288 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db46227-b9db-40e8-81dd-adb3aa018402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:14.628944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db46227-b9db-40e8-81dd-adb3aa018402 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:14.629236 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db46227-b9db-40e8-81dd-adb3aa018402 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:14.629521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db46227-b9db-40e8-81dd-adb3aa018402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:14.630358 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db46227-b9db-40e8-81dd-adb3aa018402 None admin] RBAC: Adding query filter params (role.id=0e2011aac3e1430cb96bdbf810e89eb7, scope.project.id=3366828c9f264c46a2d21ee94c2e705d, user.id=7e0b262e8410401e8d9297ca2732da45) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:14.630671 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db46227-b9db-40e8-81dd-adb3aa018402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:14.632139 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db46227-b9db-40e8-81dd-adb3aa018402 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:14.637772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 95/191] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:14 2022] GET /identity/v3/role_assignments?user.id=7e0b262e8410401e8d9297ca2732da45&scope.project.id=3366828c9f264c46a2d21ee94c2e705d&role.id=0e2011aac3e1430cb96bdbf810e89eb7 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:14.638546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db46227-b9db-40e8-81dd-adb3aa018402 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:14.661807 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132efb9-9c5c-4e57-84f9-3daba7adf4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:14.662633 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132efb9-9c5c-4e57-84f9-3daba7adf4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:14.662954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132efb9-9c5c-4e57-84f9-3daba7adf4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:14.663230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132efb9-9c5c-4e57-84f9-3daba7adf4b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:14.664485 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132efb9-9c5c-4e57-84f9-3daba7adf4b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:14.665980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132efb9-9c5c-4e57-84f9-3daba7adf4b3 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:14.667809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 97/192] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:14.668654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132efb9-9c5c-4e57-84f9-3daba7adf4b3 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:14.759068 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db46227-b9db-40e8-81dd-adb3aa018402 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:14.783874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132efb9-9c5c-4e57-84f9-3daba7adf4b3 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:14.799192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00033710-1646-4514-9c6c-9cedc55e4bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:14.800115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033710-1646-4514-9c6c-9cedc55e4bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:14.800392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033710-1646-4514-9c6c-9cedc55e4bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:14.800662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033710-1646-4514-9c6c-9cedc55e4bc9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:14.813869 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00033710-1646-4514-9c6c-9cedc55e4bc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:14.822342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-00033710-1646-4514-9c6c-9cedc55e4bc9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:14.822342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:14.822342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:14.822342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:14.822342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:14.822342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:14.822342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:14.822342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:14.822342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:14.822342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:14.822342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:14.822342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:14.822342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:14.822342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:14.822342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:14.822342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:14.823242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:14.823242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:14.823242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:14.823242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:14.823242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:14.823242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:14.823242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:14.823242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:14.823242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:14.823242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:14.823242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:14.823242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:14.823242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:14.823242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:14.823242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:14.823242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:14.824030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:14.824030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:14.824030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:14.824030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:14.824030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:14.824030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:14.824030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:14.824030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:14.824030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:14.824030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:14.824030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:14.824030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:14.824030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 96/193] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 08:59:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:14.824663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00033710-1646-4514-9c6c-9cedc55e4bc9 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:14.828886 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26cbf7-4ba4-4b61-a775-5a5028eac97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:14.830425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26cbf7-4ba4-4b61-a775-5a5028eac97f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:14.830906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26cbf7-4ba4-4b61-a775-5a5028eac97f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:14.831193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26cbf7-4ba4-4b61-a775-5a5028eac97f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:14.840474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:14.847273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:14.908210 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26cbf7-4ba4-4b61-a775-5a5028eac97f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:14.909675 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26cbf7-4ba4-4b61-a775-5a5028eac97f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:14.938789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00033710-1646-4514-9c6c-9cedc55e4bc9 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:14.973391 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cdcb6-cbfd-4d7d-85e1-1f6bb47242d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:14.974323 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cdcb6-cbfd-4d7d-85e1-1f6bb47242d4 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:14.974617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cdcb6-cbfd-4d7d-85e1-1f6bb47242d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:14.974929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cdcb6-cbfd-4d7d-85e1-1f6bb47242d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:14.985036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0cdcb6-cbfd-4d7d-85e1-1f6bb47242d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:14.992662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-3e0cdcb6-cbfd-4d7d-85e1-1f6bb47242d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:14.992662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:14.992662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:14.992662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:14.992662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:14.992662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:14.992662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:14.992662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:14.992662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:14.992662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:14.992662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:14.992662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:14.992662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:14.992662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:14.992662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:14.992662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:14.993563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:14.993563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:14.993563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:14.993563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:14.993563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:14.993563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:14.993563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:14.993563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:14.993563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:14.993563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:14.993563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:14.993563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:14.993563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:14.993563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:14.993563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:14.993563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:14.994347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:14.994347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:14.994347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:14.994347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:14.994347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:14.994347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:14.994347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:14.994347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:14.994347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:14.994347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:14.994347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:14.994347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:14.994347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 97/194] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:14.995047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cdcb6-cbfd-4d7d-85e1-1f6bb47242d4 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:15.016674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 98/195] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:15.017701 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26cbf7-4ba4-4b61-a775-5a5028eac97f None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:15.057400 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:15.071893 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:15.097420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cdcb6-cbfd-4d7d-85e1-1f6bb47242d4 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:15.116761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26cbf7-4ba4-4b61-a775-5a5028eac97f None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:15.131032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e099b-06c5-4904-8b99-535cc5990eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:15.131701 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e099b-06c5-4904-8b99-535cc5990eb8 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:15.131988 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e099b-06c5-4904-8b99-535cc5990eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:15.132270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e099b-06c5-4904-8b99-535cc5990eb8 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:15.132986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e099b-06c5-4904-8b99-535cc5990eb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:15.134442 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e099b-06c5-4904-8b99-535cc5990eb8 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:15.150239 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b381d-8397-4bb7-ad5f-e836e31395b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:15.150931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b381d-8397-4bb7-ad5f-e836e31395b4 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:15.151219 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b381d-8397-4bb7-ad5f-e836e31395b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:15.151541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b381d-8397-4bb7-ad5f-e836e31395b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:15.152292 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b381d-8397-4bb7-ad5f-e836e31395b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:15.152597 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b381d-8397-4bb7-ad5f-e836e31395b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:15.153963 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b381d-8397-4bb7-ad5f-e836e31395b4 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:15.154740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 98/196] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 08:59:14 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:59:15.155854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e099b-06c5-4904-8b99-535cc5990eb8 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:15.162531 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 99/197] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:15.163597 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b381d-8397-4bb7-ad5f-e836e31395b4 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:15.310416 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:15.316582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b381d-8397-4bb7-ad5f-e836e31395b4 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:15.319147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e099b-06c5-4904-8b99-535cc5990eb8 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:15.357015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:15.368004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230ec3a-5bb3-42df-a01f-d6f9084c128a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:15.370541 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230ec3a-5bb3-42df-a01f-d6f9084c128a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:15.370541 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230ec3a-5bb3-42df-a01f-d6f9084c128a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:15.370541 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230ec3a-5bb3-42df-a01f-d6f9084c128a None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:15.371299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230ec3a-5bb3-42df-a01f-d6f9084c128a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:15.371794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230ec3a-5bb3-42df-a01f-d6f9084c128a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:15.373727 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230ec3a-5bb3-42df-a01f-d6f9084c128a None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:15.374808 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a450c4b-e701-4cab-93fa-c628c745079e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:15.377560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a450c4b-e701-4cab-93fa-c628c745079e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:15.378005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a450c4b-e701-4cab-93fa-c628c745079e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:15.378430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a450c4b-e701-4cab-93fa-c628c745079e None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:15.380242 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a450c4b-e701-4cab-93fa-c628c745079e None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:15.381904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a450c4b-e701-4cab-93fa-c628c745079e None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:15.382805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 99/198] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:15.388005 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230ec3a-5bb3-42df-a01f-d6f9084c128a None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:15.389606 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 100/199] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:15 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:15.396996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a450c4b-e701-4cab-93fa-c628c745079e None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:15.525166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230ec3a-5bb3-42df-a01f-d6f9084c128a None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:15.537570 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a450c4b-e701-4cab-93fa-c628c745079e None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:15.572550 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aecc4e-881a-4580-bc50-c2c9090f2be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:15.573469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aecc4e-881a-4580-bc50-c2c9090f2be5 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:15.573754 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aecc4e-881a-4580-bc50-c2c9090f2be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:15.574028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aecc4e-881a-4580-bc50-c2c9090f2be5 None admin] PATH_INFO: `/v3/users/79c76db5606f4aa7a3a8c6bd45dfebf0` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:15.575677 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aecc4e-881a-4580-bc50-c2c9090f2be5 None admin] RBAC: Authorizing `identity:get_user(user_id=79c76db5606f4aa7a3a8c6bd45dfebf0)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:15.577719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92010460-34d9-459d-ae97-02e94490e91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:15.577978 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aecc4e-881a-4580-bc50-c2c9090f2be5 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:15.578609 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92010460-34d9-459d-ae97-02e94490e91a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:15.578942 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92010460-34d9-459d-ae97-02e94490e91a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:15.579223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92010460-34d9-459d-ae97-02e94490e91a None admin] PATH_INFO: `/v3/users/07188900e83d448685f5ee569bb0b6da` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:15.580440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 100/200] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:15 2022] GET /identity/v3/users/79c76db5606f4aa7a3a8c6bd45dfebf0 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:15.581313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aecc4e-881a-4580-bc50-c2c9090f2be5 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:15.581587 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92010460-34d9-459d-ae97-02e94490e91a None admin] RBAC: Authorizing `identity:get_user(user_id=07188900e83d448685f5ee569bb0b6da)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:15.583329 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92010460-34d9-459d-ae97-02e94490e91a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:15.585806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 101/201] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:15 2022] GET /identity/v3/users/07188900e83d448685f5ee569bb0b6da => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:15.586120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92010460-34d9-459d-ae97-02e94490e91a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:15.636632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:15.684731 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:15.714363 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92010460-34d9-459d-ae97-02e94490e91a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:15.745350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aecc4e-881a-4580-bc50-c2c9090f2be5 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:15.754144 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4e399-ac9f-4603-bcd3-5b030c04b2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:15.755295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4e399-ac9f-4603-bcd3-5b030c04b2ef None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:15.755579 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4e399-ac9f-4603-bcd3-5b030c04b2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:15.755860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4e399-ac9f-4603-bcd3-5b030c04b2ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:15.784995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c1d43-1451-4932-bb88-b218ee7d0599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:15.785749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c1d43-1451-4932-bb88-b218ee7d0599 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:15.786174 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c1d43-1451-4932-bb88-b218ee7d0599 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:15.786516 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c1d43-1451-4932-bb88-b218ee7d0599 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:15.787514 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c1d43-1451-4932-bb88-b218ee7d0599 None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7, user.id=79c76db5606f4aa7a3a8c6bd45dfebf0) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:15.787936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c1d43-1451-4932-bb88-b218ee7d0599 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:15.789623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c1d43-1451-4932-bb88-b218ee7d0599 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:15.796133 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 101/202] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:15 2022] GET /identity/v3/role_assignments?user.id=79c76db5606f4aa7a3a8c6bd45dfebf0&scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:15.797689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c1d43-1451-4932-bb88-b218ee7d0599 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:15.899712 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4e399-ac9f-4603-bcd3-5b030c04b2ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:15.903344 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4e399-ac9f-4603-bcd3-5b030c04b2ef None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:15.965604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:15.980010 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c1d43-1451-4932-bb88-b218ee7d0599 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:16.008570 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:16.030293 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cba413-c3ef-4416-8ccd-17009abe76c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:16.031109 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cba413-c3ef-4416-8ccd-17009abe76c9 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:16.031399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cba413-c3ef-4416-8ccd-17009abe76c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:16.031684 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cba413-c3ef-4416-8ccd-17009abe76c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:16.075453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 102/203] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:16.087692 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4e399-ac9f-4603-bcd3-5b030c04b2ef None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:16.138873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cba413-c3ef-4416-8ccd-17009abe76c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:16.140425 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cba413-c3ef-4416-8ccd-17009abe76c9 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:16.261736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4e399-ac9f-4603-bcd3-5b030c04b2ef None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:16.273897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 102/204] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:16.281458 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cba413-c3ef-4416-8ccd-17009abe76c9 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:16.302120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bafe5c-3d39-4ca5-905b-8531a3ff6d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:16.303317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bafe5c-3d39-4ca5-905b-8531a3ff6d39 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:16.303814 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bafe5c-3d39-4ca5-905b-8531a3ff6d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:16.304285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bafe5c-3d39-4ca5-905b-8531a3ff6d39 None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:16.305655 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bafe5c-3d39-4ca5-905b-8531a3ff6d39 None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:16.307316 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bafe5c-3d39-4ca5-905b-8531a3ff6d39 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:16.309247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 103/205] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:16 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:16.315738 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bafe5c-3d39-4ca5-905b-8531a3ff6d39 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:16.319862 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:16.322131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:16.406570 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cba413-c3ef-4416-8ccd-17009abe76c9 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:16.447324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e5975-30e4-4544-85a6-4be40dd89ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:16.448740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e5975-30e4-4544-85a6-4be40dd89ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:16.449986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e5975-30e4-4544-85a6-4be40dd89ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:16.451156 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e5975-30e4-4544-85a6-4be40dd89ae6 None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:16.454966 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e5975-30e4-4544-85a6-4be40dd89ae6 None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:16.456794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e5975-30e4-4544-85a6-4be40dd89ae6 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:16.459215 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 103/206] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:16 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:16.465740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e5975-30e4-4544-85a6-4be40dd89ae6 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:16.540677 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bafe5c-3d39-4ca5-905b-8531a3ff6d39 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:16.611005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d051590-faf3-4514-9f47-958d20ba703c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:16.612852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d051590-faf3-4514-9f47-958d20ba703c None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:16.613148 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d051590-faf3-4514-9f47-958d20ba703c None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:16.613433 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d051590-faf3-4514-9f47-958d20ba703c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:16.614283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d051590-faf3-4514-9f47-958d20ba703c None admin] RBAC: Adding query filter params (role.id=05ca4843c34e463c94de09f756d95318, scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7, user.id=79c76db5606f4aa7a3a8c6bd45dfebf0) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:16.614601 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d051590-faf3-4514-9f47-958d20ba703c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:16.616084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d051590-faf3-4514-9f47-958d20ba703c None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:16.631572 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 104/207] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:16 2022] GET /identity/v3/role_assignments?user.id=79c76db5606f4aa7a3a8c6bd45dfebf0&scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7&role.id=05ca4843c34e463c94de09f756d95318 => generated 259 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:16.646299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:16.667292 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e5975-30e4-4544-85a6-4be40dd89ae6 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:16.705549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2a03f-0949-40f6-b924-5e6c5b442b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:16.706225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2a03f-0949-40f6-b924-5e6c5b442b3b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:16.706820 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2a03f-0949-40f6-b924-5e6c5b442b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:16.707101 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2a03f-0949-40f6-b924-5e6c5b442b3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:16.707967 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2a03f-0949-40f6-b924-5e6c5b442b3b None admin] RBAC: Adding query filter params (role.id=559659e5dbc346d79735375b9130736d, scope.project.id=81fa154d1fc04266abee9f0051cba2fa, user.id=07188900e83d448685f5ee569bb0b6da) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:16.708280 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2a03f-0949-40f6-b924-5e6c5b442b3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:16.709776 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2a03f-0949-40f6-b924-5e6c5b442b3b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:16.716999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 104/208] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:16 2022] GET /identity/v3/role_assignments?user.id=07188900e83d448685f5ee569bb0b6da&scope.project.id=81fa154d1fc04266abee9f0051cba2fa&role.id=559659e5dbc346d79735375b9130736d => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:17.442895 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352284b5-2ed5-4bfb-89b9-16a01d21ac7f None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:17.443442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352284b5-2ed5-4bfb-89b9-16a01d21ac7f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:17.443442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352284b5-2ed5-4bfb-89b9-16a01d21ac7f None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:17.445520 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 105/209] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:17.450983 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d7d6c-e168-4407-b695-eec221809021 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:17.451388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d7d6c-e168-4407-b695-eec221809021 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:17.451575 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d7d6c-e168-4407-b695-eec221809021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:17.473833 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-032d7d6c-e168-4407-b695-eec221809021 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:17.475072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-032d7d6c-e168-4407-b695-eec221809021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:17.557788 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 105/210] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:17.581079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352284b5-2ed5-4bfb-89b9-16a01d21ac7f None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:17.648634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aca8fb-3c48-40c8-a458-21f19a11b7bb None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:17.648924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aca8fb-3c48-40c8-a458-21f19a11b7bb None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:17.649205 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aca8fb-3c48-40c8-a458-21f19a11b7bb None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:17.650586 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 106/211] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:17.655257 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d842c1c-ddc6-4f48-9765-8a0ac2a72728 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:17.655567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d842c1c-ddc6-4f48-9765-8a0ac2a72728 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:17.655849 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d842c1c-ddc6-4f48-9765-8a0ac2a72728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:17.656757 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:17.673688 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-4d842c1c-ddc6-4f48-9765-8a0ac2a72728 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:17.674896 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4d842c1c-ddc6-4f48-9765-8a0ac2a72728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:17.693260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352284b5-2ed5-4bfb-89b9-16a01d21ac7f None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:17.728591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3893e85-b608-40fd-88db-7a70c935640b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:17.729400 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3893e85-b608-40fd-88db-7a70c935640b None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:17.729684 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3893e85-b608-40fd-88db-7a70c935640b None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:17.729961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3893e85-b608-40fd-88db-7a70c935640b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:17.741172 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3893e85-b608-40fd-88db-7a70c935640b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:17.748833 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 107/212] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:17.749441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-b3893e85-b608-40fd-88db-7a70c935640b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:17.749441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:17.749441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:17.749441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:17.749441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:17.749441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:17.749441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:17.749441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:17.749441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:17.749441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:17.749441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:17.749441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:17.749441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:17.749441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:17.749441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:17.749441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:17.750523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:17.750523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:17.750523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:17.750523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:17.750523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:17.750523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:17.750523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:17.750523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:17.750523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:17.750523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:17.750523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:17.750523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:17.750523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:17.750523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:17.750523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:17.750523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:17.751342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:17.751342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:17.751342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:17.751342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:17.751342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:17.751342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:17.751342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:17.751342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:17.751342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:17.751342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:17.751342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:17.751342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:17.751342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 106/213] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:17.758015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d842c1c-ddc6-4f48-9765-8a0ac2a72728 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:17.773935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3893e85-b608-40fd-88db-7a70c935640b None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:17.778896 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:17.883198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d842c1c-ddc6-4f48-9765-8a0ac2a72728 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:17.908789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3893e85-b608-40fd-88db-7a70c935640b None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:17.922424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d5f83-8494-454e-bc0a-fdf3cfc09323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:17.923497 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d5f83-8494-454e-bc0a-fdf3cfc09323 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:17.923974 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d5f83-8494-454e-bc0a-fdf3cfc09323 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:17.924456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d5f83-8494-454e-bc0a-fdf3cfc09323 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:17.925399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d5f83-8494-454e-bc0a-fdf3cfc09323 None admin] RBAC: Adding query filter params (name=member) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:17.925898 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d5f83-8494-454e-bc0a-fdf3cfc09323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:17.927510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d5f83-8494-454e-bc0a-fdf3cfc09323 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:17.936566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 108/214] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:17.942660 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d5f83-8494-454e-bc0a-fdf3cfc09323 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:17.943464 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:17.954904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96beba-d2e9-4a43-bce1-c8c70811ed8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:17.956354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96beba-d2e9-4a43-bce1-c8c70811ed8f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:17.956652 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96beba-d2e9-4a43-bce1-c8c70811ed8f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:17.956940 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96beba-d2e9-4a43-bce1-c8c70811ed8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:17.958151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96beba-d2e9-4a43-bce1-c8c70811ed8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:17.960115 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96beba-d2e9-4a43-bce1-c8c70811ed8f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:17.962475 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 107/215] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:17.968656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96beba-d2e9-4a43-bce1-c8c70811ed8f None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:18.034892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:18.058354 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d5f83-8494-454e-bc0a-fdf3cfc09323 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:18.076523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96beba-d2e9-4a43-bce1-c8c70811ed8f None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:18.096473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c9743-6d97-4f24-91c7-bdeeef436f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:18.097582 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c9743-6d97-4f24-91c7-bdeeef436f26 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:18.098006 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c9743-6d97-4f24-91c7-bdeeef436f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:18.098422 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c9743-6d97-4f24-91c7-bdeeef436f26 None admin] PATH_INFO: `/v3/users/eec1890ca3ac431e8737395b1e8bc927` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:18.112753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7bdb3-3ad6-421a-9b98-43b50c2d91c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:18.113274 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c9743-6d97-4f24-91c7-bdeeef436f26 None admin] RBAC: Authorizing `identity:get_user(user_id=eec1890ca3ac431e8737395b1e8bc927)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:18.113586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7bdb3-3ad6-421a-9b98-43b50c2d91c5 None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:18.113943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7bdb3-3ad6-421a-9b98-43b50c2d91c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:18.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7bdb3-3ad6-421a-9b98-43b50c2d91c5 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:18.115129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7bdb3-3ad6-421a-9b98-43b50c2d91c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:18.115743 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c9743-6d97-4f24-91c7-bdeeef436f26 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:18.116688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7bdb3-3ad6-421a-9b98-43b50c2d91c5 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:18.117792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 109/216] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:17 2022] GET /identity/v3/users/eec1890ca3ac431e8737395b1e8bc927 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:18.123177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c9743-6d97-4f24-91c7-bdeeef436f26 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:18.154703 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 108/217] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 08:59:17 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:59:18.247588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c9743-6d97-4f24-91c7-bdeeef436f26 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:18.292469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadf72b-e6a5-4f24-8fcd-3ddc61e38d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:18.293203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadf72b-e6a5-4f24-8fcd-3ddc61e38d47 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:18.293487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadf72b-e6a5-4f24-8fcd-3ddc61e38d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:18.293784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadf72b-e6a5-4f24-8fcd-3ddc61e38d47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:18.299409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:18.317125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f477c-616e-467e-801f-c1703d616f02 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:18.317418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f477c-616e-467e-801f-c1703d616f02 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:18.317722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f477c-616e-467e-801f-c1703d616f02 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:18.321023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 109/218] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:59:18.327929 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b320d-9ec1-4125-a7ea-3cc53bee4908 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:18.328266 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b320d-9ec1-4125-a7ea-3cc53bee4908 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:18.328508 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b320d-9ec1-4125-a7ea-3cc53bee4908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:18.347492 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-5a1b320d-9ec1-4125-a7ea-3cc53bee4908 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:18.348688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5a1b320d-9ec1-4125-a7ea-3cc53bee4908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:18.380545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dadf72b-e6a5-4f24-8fcd-3ddc61e38d47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:18.382114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dadf72b-e6a5-4f24-8fcd-3ddc61e38d47 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:18.412593 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:18.437453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 110/219] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:18.460913 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b320d-9ec1-4125-a7ea-3cc53bee4908 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:18.505732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 110/220] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 08:59:18.510993 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadf72b-e6a5-4f24-8fcd-3ddc61e38d47 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:18.574090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:18.586838 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b320d-9ec1-4125-a7ea-3cc53bee4908 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:18.633054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db787328-b220-4806-af69-68206fc59cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:18.634006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db787328-b220-4806-af69-68206fc59cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:18.634414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db787328-b220-4806-af69-68206fc59cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:18.634892 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db787328-b220-4806-af69-68206fc59cd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:18.647140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db787328-b220-4806-af69-68206fc59cd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:18.656558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-db787328-b220-4806-af69-68206fc59cd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:18.656558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:18.656558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:18.656558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:18.656558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:18.656558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:18.656558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:18.656558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:18.656558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:18.656558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:18.656558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:18.656558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:18.656558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:18.656558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:18.656558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:18.656558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:18.657710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:18.657710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:18.657710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:18.657710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:18.657710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:18.657710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:18.657710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:18.657710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:18.657710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:18.657710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:18.657710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:18.657710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:18.657710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:18.657710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:18.657710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:18.657710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:18.658660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:18.658660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:18.658660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:18.658660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:18.658660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:18.658660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:18.658660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:18.658660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:18.658660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:18.658660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:18.658660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:18.658660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:18.660178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 111/221] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 08:59:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:18.665795 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadf72b-e6a5-4f24-8fcd-3ddc61e38d47 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:18.666534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db787328-b220-4806-af69-68206fc59cd5 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:18.713353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ef8df-7306-47e4-8407-1d333257672e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:18.714329 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ef8df-7306-47e4-8407-1d333257672e None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:18.714701 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ef8df-7306-47e4-8407-1d333257672e None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:18.715094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ef8df-7306-47e4-8407-1d333257672e None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:18.716528 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ef8df-7306-47e4-8407-1d333257672e None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:18.717597 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:18.720321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ef8df-7306-47e4-8407-1d333257672e None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:18.723318 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 111/222] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:18 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:18.729484 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ef8df-7306-47e4-8407-1d333257672e None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:18.786198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db787328-b220-4806-af69-68206fc59cd5 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:18.823403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcd1bd-3edb-4a51-a050-c7b3dcfadda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:18.824484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcd1bd-3edb-4a51-a050-c7b3dcfadda1 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:18.825008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcd1bd-3edb-4a51-a050-c7b3dcfadda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:18.825504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcd1bd-3edb-4a51-a050-c7b3dcfadda1 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:18.826459 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcd1bd-3edb-4a51-a050-c7b3dcfadda1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:18.826996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcd1bd-3edb-4a51-a050-c7b3dcfadda1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:18.829235 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfcd1bd-3edb-4a51-a050-c7b3dcfadda1 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:18.838911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 112/223] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:18.845609 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcd1bd-3edb-4a51-a050-c7b3dcfadda1 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:18.898710 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ef8df-7306-47e4-8407-1d333257672e None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:18.903066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:18.953608 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7905219-a3ee-46a4-bca5-7f5322e0adc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:18.954704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7905219-a3ee-46a4-bca5-7f5322e0adc6 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:18.955777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7905219-a3ee-46a4-bca5-7f5322e0adc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:18.956261 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7905219-a3ee-46a4-bca5-7f5322e0adc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:18.957761 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7905219-a3ee-46a4-bca5-7f5322e0adc6 None admin] RBAC: Adding query filter params (role.id=0e2011aac3e1430cb96bdbf810e89eb7, scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7, user.id=eec1890ca3ac431e8737395b1e8bc927) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:18.958281 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7905219-a3ee-46a4-bca5-7f5322e0adc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:18.960654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7905219-a3ee-46a4-bca5-7f5322e0adc6 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:18.969037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 112/224] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:18 2022] GET /identity/v3/role_assignments?user.id=eec1890ca3ac431e8737395b1e8bc927&scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7&role.id=0e2011aac3e1430cb96bdbf810e89eb7 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:18.971062 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36979f95-e479-444c-8d23-b66ecfbcf4d6 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:18.971820 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36979f95-e479-444c-8d23-b66ecfbcf4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:18.972805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36979f95-e479-444c-8d23-b66ecfbcf4d6 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:18.979203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 113/225] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:18.986917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc710f8-838c-4367-b37f-0c14cf08f469 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:18.987239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc710f8-838c-4367-b37f-0c14cf08f469 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:18.987496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc710f8-838c-4367-b37f-0c14cf08f469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:18.990493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcd1bd-3edb-4a51-a050-c7b3dcfadda1 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:18.993830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:19.030799 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-6bc710f8-838c-4367-b37f-0c14cf08f469 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:19.032414 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6bc710f8-838c-4367-b37f-0c14cf08f469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:19.035348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66586fd5-7aeb-4658-9f1a-c7434f2cf3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:19.036633 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66586fd5-7aeb-4658-9f1a-c7434f2cf3ac None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:19.037120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66586fd5-7aeb-4658-9f1a-c7434f2cf3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:19.037593 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66586fd5-7aeb-4658-9f1a-c7434f2cf3ac None admin] PATH_INFO: `/v3/users/79c76db5606f4aa7a3a8c6bd45dfebf0` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:19.039216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66586fd5-7aeb-4658-9f1a-c7434f2cf3ac None admin] RBAC: Authorizing `identity:get_user(user_id=79c76db5606f4aa7a3a8c6bd45dfebf0)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:19.040989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66586fd5-7aeb-4658-9f1a-c7434f2cf3ac None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:19.049637 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 113/226] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:18 2022] GET /identity/v3/users/79c76db5606f4aa7a3a8c6bd45dfebf0 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:19.051499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66586fd5-7aeb-4658-9f1a-c7434f2cf3ac None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:19.172645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 114/227] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:19.197057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc710f8-838c-4367-b37f-0c14cf08f469 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:19.206515 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66586fd5-7aeb-4658-9f1a-c7434f2cf3ac None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:19.265961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f12640-ef4f-4c9a-9765-de29b2e3e10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:19.266862 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f12640-ef4f-4c9a-9765-de29b2e3e10f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:19.267147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f12640-ef4f-4c9a-9765-de29b2e3e10f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:19.267430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f12640-ef4f-4c9a-9765-de29b2e3e10f None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:19.269504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f12640-ef4f-4c9a-9765-de29b2e3e10f None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:19.270975 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f12640-ef4f-4c9a-9765-de29b2e3e10f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:19.272701 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 114/228] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:19 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:19.279285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f12640-ef4f-4c9a-9765-de29b2e3e10f None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:19.352803 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:19.366072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:19.373703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc710f8-838c-4367-b37f-0c14cf08f469 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:19.413586 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c2a90-379e-47f6-bc60-499f12b32765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:19.414406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c2a90-379e-47f6-bc60-499f12b32765 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:19.414715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c2a90-379e-47f6-bc60-499f12b32765 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:19.415005 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c2a90-379e-47f6-bc60-499f12b32765 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:19.426930 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c2a90-379e-47f6-bc60-499f12b32765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:19.440299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-ab4c2a90-379e-47f6-bc60-499f12b32765 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:19.440299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:19.440299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:19.440299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:19.440299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:19.440299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:19.440299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:19.440299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:19.440299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:19.440299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:19.440299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:19.440299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:19.440299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:19.440299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:19.440299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:19.440299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:19.441251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:19.441251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:19.441251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:19.441251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:19.441251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:19.441251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:19.441251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:19.441251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:19.441251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:19.441251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:19.441251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:19.441251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:19.441251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:19.441251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:19.441251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:19.441251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:19.442026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:19.442026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:19.442026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:19.442026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:19.442026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:19.442026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:19.442026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:19.442026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:19.442026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:19.442026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:19.442026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:19.442026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:19.442632 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 115/229] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 08:59:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:19.448519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c2a90-379e-47f6-bc60-499f12b32765 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:19.486952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f12640-ef4f-4c9a-9765-de29b2e3e10f None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:19.540055 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315775c2-e74a-434d-8439-e23c8d6271a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:19.540977 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315775c2-e74a-434d-8439-e23c8d6271a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:19.541267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315775c2-e74a-434d-8439-e23c8d6271a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:19.549639 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315775c2-e74a-434d-8439-e23c8d6271a1 None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7/users/79c76db5606f4aa7a3a8c6bd45dfebf0/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:19.555510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315775c2-e74a-434d-8439-e23c8d6271a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7, user_id=79c76db5606f4aa7a3a8c6bd45dfebf0, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:19.556985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315775c2-e74a-434d-8439-e23c8d6271a1 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:19.571851 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c2a90-379e-47f6-bc60-499f12b32765 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:19.580269 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 115/230] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 08:59:19 2022] PUT /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7/users/79c76db5606f4aa7a3a8c6bd45dfebf0/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:59:19.616833 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2221297-6478-4cb8-b914-e1347b959099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:19.617500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2221297-6478-4cb8-b914-e1347b959099 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:19.617792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2221297-6478-4cb8-b914-e1347b959099 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:19.618074 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2221297-6478-4cb8-b914-e1347b959099 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:19.618943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2221297-6478-4cb8-b914-e1347b959099 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:19.619257 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2221297-6478-4cb8-b914-e1347b959099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:19.620731 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2221297-6478-4cb8-b914-e1347b959099 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:19.632134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 116/231] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:19.639606 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315775c2-e74a-434d-8439-e23c8d6271a1 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:19.766804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:19.769950 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315775c2-e74a-434d-8439-e23c8d6271a1 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:19.806376 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52b597-1430-42fb-828d-e9a77dec353e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:19.807364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52b597-1430-42fb-828d-e9a77dec353e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:19.807664 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52b597-1430-42fb-828d-e9a77dec353e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:19.807948 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52b597-1430-42fb-828d-e9a77dec353e None admin] PATH_INFO: `/v3/users/79c76db5606f4aa7a3a8c6bd45dfebf0` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:19.809171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52b597-1430-42fb-828d-e9a77dec353e None admin] RBAC: Authorizing `identity:get_user(user_id=79c76db5606f4aa7a3a8c6bd45dfebf0)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:19.810623 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52b597-1430-42fb-828d-e9a77dec353e None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:19.812976 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 116/232] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:19 2022] GET /identity/v3/users/79c76db5606f4aa7a3a8c6bd45dfebf0 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:19.819094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2221297-6478-4cb8-b914-e1347b959099 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:19.845690 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:19.938021 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2221297-6478-4cb8-b914-e1347b959099 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:19.984555 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111dbc5a-2265-447d-b50a-8ae078aa12f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:19.985667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dbc5a-2265-447d-b50a-8ae078aa12f5 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:19.986207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dbc5a-2265-447d-b50a-8ae078aa12f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:19.986810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dbc5a-2265-447d-b50a-8ae078aa12f5 None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:19.988708 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111dbc5a-2265-447d-b50a-8ae078aa12f5 None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:19.991082 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111dbc5a-2265-447d-b50a-8ae078aa12f5 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:19.992934 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 117/233] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:19 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:19.998891 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52b597-1430-42fb-828d-e9a77dec353e None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:20.115969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52b597-1430-42fb-828d-e9a77dec353e None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:20.160710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c05cf-19d9-4487-8648-f266de424c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:20.161836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c05cf-19d9-4487-8648-f266de424c27 None admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:20.162264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c05cf-19d9-4487-8648-f266de424c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:20.162700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c05cf-19d9-4487-8648-f266de424c27 None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7/users/79c76db5606f4aa7a3a8c6bd45dfebf0/roles/05ca4843c34e463c94de09f756d95318` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:20.165299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c05cf-19d9-4487-8648-f266de424c27 None admin] RBAC: Authorizing `identity:create_grant(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7, user_id=79c76db5606f4aa7a3a8c6bd45dfebf0, role_id=05ca4843c34e463c94de09f756d95318)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:20.166908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c05cf-19d9-4487-8648-f266de424c27 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:20.198848 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 117/234] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 08:59:19 2022] PUT /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7/users/79c76db5606f4aa7a3a8c6bd45dfebf0/roles/05ca4843c34e463c94de09f756d95318 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:59:20.884412 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b90595-5d87-4e79-a8e6-f3ca11c22e07 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:20.885289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b90595-5d87-4e79-a8e6-f3ca11c22e07 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:20.885806 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b90595-5d87-4e79-a8e6-f3ca11c22e07 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:20.887417 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 118/235] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:20.897093 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ceadc-f063-4b69-8d08-e9a87e195492 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:20.898033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ceadc-f063-4b69-8d08-e9a87e195492 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:20.898523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ceadc-f063-4b69-8d08-e9a87e195492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:20.920251 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-e74ceadc-f063-4b69-8d08-e9a87e195492 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:20.921772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e74ceadc-f063-4b69-8d08-e9a87e195492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:20.945922 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:20.995201 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 118/236] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:21.040130 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b90595-5d87-4e79-a8e6-f3ca11c22e07 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:21.113890 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e7387-7556-4265-875a-186dc23daba8 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:21.114518 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e7387-7556-4265-875a-186dc23daba8 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:21.115030 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e7387-7556-4265-875a-186dc23daba8 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:21.116605 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 119/237] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:59:21.121459 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01105807-aa4e-44d3-b79f-58fc312a3ff7 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:21.121985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01105807-aa4e-44d3-b79f-58fc312a3ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:21.122457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01105807-aa4e-44d3-b79f-58fc312a3ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:21.151052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-01105807-aa4e-44d3-b79f-58fc312a3ff7 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:21.154163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-01105807-aa4e-44d3-b79f-58fc312a3ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:21.172178 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:21.187571 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b90595-5d87-4e79-a8e6-f3ca11c22e07 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:21.228366 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf775e-f0f5-4b9f-aa01-fc68fd7301be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:21.229562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf775e-f0f5-4b9f-aa01-fc68fd7301be None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:21.230042 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf775e-f0f5-4b9f-aa01-fc68fd7301be None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:21.230512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf775e-f0f5-4b9f-aa01-fc68fd7301be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:21.250604 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf775e-f0f5-4b9f-aa01-fc68fd7301be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:21.263189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-7baf775e-f0f5-4b9f-aa01-fc68fd7301be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:21.263189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:21.263189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:21.263189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:21.263189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:21.263189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:21.263189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:21.263189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:21.263189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:21.263189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:21.263189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:21.263189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:21.263189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:21.263189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:21.263189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:21.263189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:21.264120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:21.264120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:21.264120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:21.264120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:21.264120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:21.264120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:21.264120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:21.264120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:21.264120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:21.264120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:21.264120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:21.264120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:21.264120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:21.264120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:21.264120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:21.264120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:21.265168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:21.265168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:21.265168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:21.265168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:21.265168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:21.265168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:21.265168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:21.265168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:21.265168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:21.265168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:21.265168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:21.265168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:21.268554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 119/238] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:21.275188 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf775e-f0f5-4b9f-aa01-fc68fd7301be None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:21.304026 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 120/239] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:21.341943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01105807-aa4e-44d3-b79f-58fc312a3ff7 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:21.460896 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf775e-f0f5-4b9f-aa01-fc68fd7301be None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:21.467301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:21.482285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01105807-aa4e-44d3-b79f-58fc312a3ff7 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:21.527777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6c30d-3bae-4f7f-8862-f53985ccd488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:21.528446 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6c30d-3bae-4f7f-8862-f53985ccd488 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:21.528733 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6c30d-3bae-4f7f-8862-f53985ccd488 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:21.529012 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6c30d-3bae-4f7f-8862-f53985ccd488 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:21.529768 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6c30d-3bae-4f7f-8862-f53985ccd488 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:21.530077 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6c30d-3bae-4f7f-8862-f53985ccd488 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:21.532076 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6c30d-3bae-4f7f-8862-f53985ccd488 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:21.536812 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad27b32-847c-4eb6-bf8a-9030c31bdfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:21.537857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad27b32-847c-4eb6-bf8a-9030c31bdfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:21.538331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad27b32-847c-4eb6-bf8a-9030c31bdfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:21.538854 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad27b32-847c-4eb6-bf8a-9030c31bdfb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:21.540220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad27b32-847c-4eb6-bf8a-9030c31bdfb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:21.541837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad27b32-847c-4eb6-bf8a-9030c31bdfb1 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:21.544363 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 121/240] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:21.549959 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdba1da-b5e7-49ae-9ec0-809906687156 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:21.549959 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdba1da-b5e7-49ae-9ec0-809906687156 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:21.549959 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdba1da-b5e7-49ae-9ec0-809906687156 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:21.550822 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 120/241] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:21.555655 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 122/242] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:21 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:21.558722 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6c30d-3bae-4f7f-8862-f53985ccd488 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:21.560208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdba1da-b5e7-49ae-9ec0-809906687156 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:21.573940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:21.682346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6c30d-3bae-4f7f-8862-f53985ccd488 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:21.687192 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdba1da-b5e7-49ae-9ec0-809906687156 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:21.719712 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a973e-a4ba-4470-850d-f6033fee667f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:21.720397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a973e-a4ba-4470-850d-f6033fee667f None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:21.720835 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a973e-a4ba-4470-850d-f6033fee667f None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:21.721181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a973e-a4ba-4470-850d-f6033fee667f None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:21.722089 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a973e-a4ba-4470-850d-f6033fee667f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:21.723799 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a973e-a4ba-4470-850d-f6033fee667f None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:21.729250 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b300249-e35e-4681-ba78-b2f9d768164f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:21.730130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b300249-e35e-4681-ba78-b2f9d768164f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:21.730412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b300249-e35e-4681-ba78-b2f9d768164f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:21.730702 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b300249-e35e-4681-ba78-b2f9d768164f None admin] PATH_INFO: `/v3/users/00e0957363bd4c38a7bba6d78abd168e` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:21.751060 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 121/243] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 08:59:21 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:59:21.752618 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb215c9-17ce-46fc-9672-bd32ba9fc600 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:21.752914 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb215c9-17ce-46fc-9672-bd32ba9fc600 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:21.753251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb215c9-17ce-46fc-9672-bd32ba9fc600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:21.755325 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b300249-e35e-4681-ba78-b2f9d768164f None admin] RBAC: Authorizing `identity:get_user(user_id=00e0957363bd4c38a7bba6d78abd168e)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:21.756952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b300249-e35e-4681-ba78-b2f9d768164f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:21.758900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 123/244] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:21 2022] GET /identity/v3/users/00e0957363bd4c38a7bba6d78abd168e => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:21.768951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b300249-e35e-4681-ba78-b2f9d768164f None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:21.797230 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-bbb215c9-17ce-46fc-9672-bd32ba9fc600 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:21.798815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-bbb215c9-17ce-46fc-9672-bd32ba9fc600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:21.805152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:21.880812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:21.888800 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 122/245] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:21.946530 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb215c9-17ce-46fc-9672-bd32ba9fc600 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:21.960820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b300249-e35e-4681-ba78-b2f9d768164f None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:21.997810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e71be-c509-41a4-846f-222a735a4286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:21.998390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e71be-c509-41a4-846f-222a735a4286 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:21.998670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e71be-c509-41a4-846f-222a735a4286 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:21.998970 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e71be-c509-41a4-846f-222a735a4286 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:22.058198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb215c9-17ce-46fc-9672-bd32ba9fc600 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:22.071400 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e71be-c509-41a4-846f-222a735a4286 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:22.072868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e71be-c509-41a4-846f-222a735a4286 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:22.088537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:22.098325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c235b-f446-4092-b484-0282d7a1024d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:22.099494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c235b-f446-4092-b484-0282d7a1024d None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:22.100068 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c235b-f446-4092-b484-0282d7a1024d None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:22.100566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c235b-f446-4092-b484-0282d7a1024d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:22.112933 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c235b-f446-4092-b484-0282d7a1024d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:22.121193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-aa7c235b-f446-4092-b484-0282d7a1024d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:22.121193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:22.121193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:22.121193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:22.121193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:22.121193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:22.121193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:22.121193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:22.121193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:22.121193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:22.121193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:22.121193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:22.121193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:22.121193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:22.121193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:22.121193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:22.122079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:22.122079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:22.122079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:22.122079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:22.122079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:22.122079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:22.122079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:22.122079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:22.122079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:22.122079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:22.122079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:22.122079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:22.122079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:22.122079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:22.122079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:22.122079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:22.122981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:22.122981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:22.122981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:22.122981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:22.122981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:22.122981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:22.122981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:22.122981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:22.122981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:22.122981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:22.122981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:22.122981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:22.126213 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 123/246] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:22.132277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839cf18-a8bf-4c14-8e14-4befecb2dfe9 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:22.132557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839cf18-a8bf-4c14-8e14-4befecb2dfe9 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:22.132855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839cf18-a8bf-4c14-8e14-4befecb2dfe9 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:22.134308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 124/247] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:22.135460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839cf18-a8bf-4c14-8e14-4befecb2dfe9 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:22.187968 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 124/248] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:22.189166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0082e-4352-482e-8226-7841aa90a8c6 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:22.189459 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0082e-4352-482e-8226-7841aa90a8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:22.189759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0082e-4352-482e-8226-7841aa90a8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:22.206823 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:22.214763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-35c0082e-4352-482e-8226-7841aa90a8c6 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:22.215936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-35c0082e-4352-482e-8226-7841aa90a8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:22.268723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839cf18-a8bf-4c14-8e14-4befecb2dfe9 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:22.323208 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c63c67-fa1f-49f4-9b0b-99f704bbefa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:22.323884 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c63c67-fa1f-49f4-9b0b-99f704bbefa2 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:22.324163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c63c67-fa1f-49f4-9b0b-99f704bbefa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:22.324435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c63c67-fa1f-49f4-9b0b-99f704bbefa2 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:22.325180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c63c67-fa1f-49f4-9b0b-99f704bbefa2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:22.325484 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c63c67-fa1f-49f4-9b0b-99f704bbefa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:22.326913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c63c67-fa1f-49f4-9b0b-99f704bbefa2 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:22.329940 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 125/249] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:22.331540 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0082e-4352-482e-8226-7841aa90a8c6 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:22.337485 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 125/250] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:22.344234 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c63c67-fa1f-49f4-9b0b-99f704bbefa2 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:22.393987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:22.464521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c63c67-fa1f-49f4-9b0b-99f704bbefa2 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:22.470931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0082e-4352-482e-8226-7841aa90a8c6 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:22.486868 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:22.512262 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63dd99-5722-472f-a0fd-9dd09b70651b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:22.513694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63dd99-5722-472f-a0fd-9dd09b70651b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:22.514119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63dd99-5722-472f-a0fd-9dd09b70651b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:22.514413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116578b-e34f-4a73-a46e-bbf7c394d14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:22.514743 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63dd99-5722-472f-a0fd-9dd09b70651b None admin] PATH_INFO: `/v3/users/eec1890ca3ac431e8737395b1e8bc927` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:22.515720 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116578b-e34f-4a73-a46e-bbf7c394d14a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:22.516103 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116578b-e34f-4a73-a46e-bbf7c394d14a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:22.516475 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116578b-e34f-4a73-a46e-bbf7c394d14a None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:22.516774 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63dd99-5722-472f-a0fd-9dd09b70651b None admin] RBAC: Authorizing `identity:get_user(user_id=eec1890ca3ac431e8737395b1e8bc927)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:22.517860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116578b-e34f-4a73-a46e-bbf7c394d14a None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:22.518663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63dd99-5722-472f-a0fd-9dd09b70651b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:22.519577 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116578b-e34f-4a73-a46e-bbf7c394d14a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:22.520967 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 126/251] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:22 2022] GET /identity/v3/users/eec1890ca3ac431e8737395b1e8bc927 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:22.521566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 126/252] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:22 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:22.522228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63dd99-5722-472f-a0fd-9dd09b70651b None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:22.527834 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116578b-e34f-4a73-a46e-bbf7c394d14a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:22.643953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63dd99-5722-472f-a0fd-9dd09b70651b None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:22.651353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116578b-e34f-4a73-a46e-bbf7c394d14a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:22.679558 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b022d95a-547a-472f-b212-c33d3ed58058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:22.680590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022d95a-547a-472f-b212-c33d3ed58058 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:22.680932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022d95a-547a-472f-b212-c33d3ed58058 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:22.681294 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022d95a-547a-472f-b212-c33d3ed58058 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:22.688261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:22.692236 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b022d95a-547a-472f-b212-c33d3ed58058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:22.702702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-b022d95a-547a-472f-b212-c33d3ed58058 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:22.702702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:22.702702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:22.702702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:22.702702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:22.702702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:22.702702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:22.702702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:22.702702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:22.702702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:22.702702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:22.702702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:22.702702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:22.702702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:22.702702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:22.702702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:22.703783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:22.703783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:22.703783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:22.703783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:22.703783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:22.703783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:22.703783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:22.703783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:22.703783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:22.703783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:22.703783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:22.703783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:22.703783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:22.703783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:22.703783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:22.703783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:22.704755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:22.704755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:22.704755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:22.704755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:22.704755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:22.704755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:22.704755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:22.704755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:22.704755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:22.704755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:22.704755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:22.704755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:22.704755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 127/253] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 08:59:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:22.704755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b022d95a-547a-472f-b212-c33d3ed58058 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:22.724363 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb00879-d1fa-4492-89ec-45d1d2a51ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:22.725453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb00879-d1fa-4492-89ec-45d1d2a51ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:22.725893 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb00879-d1fa-4492-89ec-45d1d2a51ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:22.726309 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb00879-d1fa-4492-89ec-45d1d2a51ef6 None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:22.727649 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb00879-d1fa-4492-89ec-45d1d2a51ef6 None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:22.729229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb00879-d1fa-4492-89ec-45d1d2a51ef6 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:22.731152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 127/254] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:22 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:22.732138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb00879-d1fa-4492-89ec-45d1d2a51ef6 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:22.779134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:22.843628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb00879-d1fa-4492-89ec-45d1d2a51ef6 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:22.882160 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65c204-f71b-4179-ad50-f2c7fbc9dceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:22.883316 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65c204-f71b-4179-ad50-f2c7fbc9dceb None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:22.883916 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65c204-f71b-4179-ad50-f2c7fbc9dceb None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:22.884407 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65c204-f71b-4179-ad50-f2c7fbc9dceb None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:22.885353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65c204-f71b-4179-ad50-f2c7fbc9dceb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:22.885860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65c204-f71b-4179-ad50-f2c7fbc9dceb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:22.887580 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65c204-f71b-4179-ad50-f2c7fbc9dceb None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:22.896652 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 128/255] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:22.897781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65c204-f71b-4179-ad50-f2c7fbc9dceb None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:22.902178 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b022d95a-547a-472f-b212-c33d3ed58058 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:22.955070 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:22.968906 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7054b3c-3edd-4333-9483-5b41fcb5c882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:22.969455 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7054b3c-3edd-4333-9483-5b41fcb5c882 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:22.969721 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7054b3c-3edd-4333-9483-5b41fcb5c882 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:22.969972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7054b3c-3edd-4333-9483-5b41fcb5c882 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:22.970851 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7054b3c-3edd-4333-9483-5b41fcb5c882 None admin] RBAC: Adding query filter params (role.id=559659e5dbc346d79735375b9130736d, scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7, user.id=00e0957363bd4c38a7bba6d78abd168e) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:22.971167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7054b3c-3edd-4333-9483-5b41fcb5c882 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:22.972633 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7054b3c-3edd-4333-9483-5b41fcb5c882 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:22.978757 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 128/256] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:22 2022] GET /identity/v3/role_assignments?user.id=00e0957363bd4c38a7bba6d78abd168e&scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7&role.id=559659e5dbc346d79735375b9130736d => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:22.979580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7054b3c-3edd-4333-9483-5b41fcb5c882 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:23.012417 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65c204-f71b-4179-ad50-f2c7fbc9dceb None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:23.052125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca215ebf-6d8e-4d11-af02-df3b7edebca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:23.053391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca215ebf-6d8e-4d11-af02-df3b7edebca2 None admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:23.053874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca215ebf-6d8e-4d11-af02-df3b7edebca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:23.054398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca215ebf-6d8e-4d11-af02-df3b7edebca2 None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7/users/eec1890ca3ac431e8737395b1e8bc927/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:23.056790 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca215ebf-6d8e-4d11-af02-df3b7edebca2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7, user_id=eec1890ca3ac431e8737395b1e8bc927, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:23.058501 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca215ebf-6d8e-4d11-af02-df3b7edebca2 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:23.075247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 129/257] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 08:59:22 2022] PUT /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7/users/eec1890ca3ac431e8737395b1e8bc927/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:59:23.077926 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5bbe4-8b00-4760-b1a1-821efbd6120a None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:23.078516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5bbe4-8b00-4760-b1a1-821efbd6120a None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:23.079185 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5bbe4-8b00-4760-b1a1-821efbd6120a None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:23.081134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 130/258] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:23.088890 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccd66f-54fa-4ccf-85b2-232d2e728a3d None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:23.089840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccd66f-54fa-4ccf-85b2-232d2e728a3d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:23.090314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccd66f-54fa-4ccf-85b2-232d2e728a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:23.096041 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7054b3c-3edd-4333-9483-5b41fcb5c882 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:23.108337 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:23.122266 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-f3ccd66f-54fa-4ccf-85b2-232d2e728a3d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:23.124080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f3ccd66f-54fa-4ccf-85b2-232d2e728a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:23.143628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703028fb-9c4b-4e44-9d04-e23491e5bd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:23.144535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703028fb-9c4b-4e44-9d04-e23491e5bd61 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:23.144811 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703028fb-9c4b-4e44-9d04-e23491e5bd61 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:23.145108 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703028fb-9c4b-4e44-9d04-e23491e5bd61 None admin] PATH_INFO: `/v3/users/79c76db5606f4aa7a3a8c6bd45dfebf0` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:23.146669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703028fb-9c4b-4e44-9d04-e23491e5bd61 None admin] RBAC: Authorizing `identity:get_user(user_id=79c76db5606f4aa7a3a8c6bd45dfebf0)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:23.147921 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703028fb-9c4b-4e44-9d04-e23491e5bd61 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:23.149897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 129/259] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:22 2022] GET /identity/v3/users/79c76db5606f4aa7a3a8c6bd45dfebf0 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:23.155903 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703028fb-9c4b-4e44-9d04-e23491e5bd61 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:23.212928 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 131/260] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:23.270547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccd66f-54fa-4ccf-85b2-232d2e728a3d None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:23.322623 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:23.340157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703028fb-9c4b-4e44-9d04-e23491e5bd61 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:23.377919 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3198887-fb80-456f-a273-9ecffc6f5d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:23.378641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3198887-fb80-456f-a273-9ecffc6f5d1a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:23.383052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3198887-fb80-456f-a273-9ecffc6f5d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:23.383052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3198887-fb80-456f-a273-9ecffc6f5d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:23.424815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccd66f-54fa-4ccf-85b2-232d2e728a3d None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:23.443060 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:23.469391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffee119-d838-48cf-bad7-cbc030e323db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:23.470778 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffee119-d838-48cf-bad7-cbc030e323db None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:23.471066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffee119-d838-48cf-bad7-cbc030e323db None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:23.471336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffee119-d838-48cf-bad7-cbc030e323db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:23.473780 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3198887-fb80-456f-a273-9ecffc6f5d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:23.475361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3198887-fb80-456f-a273-9ecffc6f5d1a None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:23.503888 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffee119-d838-48cf-bad7-cbc030e323db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:23.518336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-3ffee119-d838-48cf-bad7-cbc030e323db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:23.518336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:23.518336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:23.518336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:23.518336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:23.518336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:23.518336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:23.518336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:23.518336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:23.518336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:23.518336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:23.518336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:23.518336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:23.518336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:23.518336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:23.518336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:23.519431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:23.519431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:23.519431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:23.519431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:23.519431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:23.519431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:23.519431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:23.519431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:23.519431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:23.519431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:23.519431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:23.519431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:23.519431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:23.519431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:23.519431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:23.519431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:23.520676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:23.520676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:23.520676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:23.520676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:23.520676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:23.520676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:23.520676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:23.520676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:23.520676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:23.520676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:23.520676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:23.520676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:23.520676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 132/261] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 08:59:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:23.550746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffee119-d838-48cf-bad7-cbc030e323db None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:23.596230 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 130/262] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 08:59:23.601632 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3198887-fb80-456f-a273-9ecffc6f5d1a None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:23.685500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:23.709127 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3198887-fb80-456f-a273-9ecffc6f5d1a None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:23.731025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:23.747370 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63b2dd-12e8-4ade-b213-7c5264efb388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:23.748599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63b2dd-12e8-4ade-b213-7c5264efb388 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:23.749185 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63b2dd-12e8-4ade-b213-7c5264efb388 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:23.750192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c63b2dd-12e8-4ade-b213-7c5264efb388 None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:23.750610 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffee119-d838-48cf-bad7-cbc030e323db None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:23.751878 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63b2dd-12e8-4ade-b213-7c5264efb388 None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:23.753783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c63b2dd-12e8-4ade-b213-7c5264efb388 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:23.755825 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 131/263] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:23 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:23.761660 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63b2dd-12e8-4ade-b213-7c5264efb388 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:23.814657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90005f8f-abd8-446d-a34d-d45f576375e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:23.815324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90005f8f-abd8-446d-a34d-d45f576375e5 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:23.815611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90005f8f-abd8-446d-a34d-d45f576375e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:23.815890 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90005f8f-abd8-446d-a34d-d45f576375e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:23.816727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90005f8f-abd8-446d-a34d-d45f576375e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:23.817016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90005f8f-abd8-446d-a34d-d45f576375e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:23.818419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90005f8f-abd8-446d-a34d-d45f576375e5 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:23.835145 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 133/264] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:23.844213 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90005f8f-abd8-446d-a34d-d45f576375e5 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:23.878748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c63b2dd-12e8-4ade-b213-7c5264efb388 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:23.917067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c732b1-8b3e-4847-97c7-8c6fee3ce9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:23.918696 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c732b1-8b3e-4847-97c7-8c6fee3ce9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:23.919189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c732b1-8b3e-4847-97c7-8c6fee3ce9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:23.920536 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c732b1-8b3e-4847-97c7-8c6fee3ce9b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:23.922398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c732b1-8b3e-4847-97c7-8c6fee3ce9b0 None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7, user.id=79c76db5606f4aa7a3a8c6bd45dfebf0) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:23.923110 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c732b1-8b3e-4847-97c7-8c6fee3ce9b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:23.926987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c732b1-8b3e-4847-97c7-8c6fee3ce9b0 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:23.937936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 132/265] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:23 2022] GET /identity/v3/role_assignments?user.id=79c76db5606f4aa7a3a8c6bd45dfebf0&scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:24.012542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90005f8f-abd8-446d-a34d-d45f576375e5 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:24.067885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f053d44-3090-49bb-8f4d-41aae8031104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:24.069475 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f053d44-3090-49bb-8f4d-41aae8031104 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:24.069973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f053d44-3090-49bb-8f4d-41aae8031104 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:24.070452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f053d44-3090-49bb-8f4d-41aae8031104 None admin] PATH_INFO: `/v3/users/79c76db5606f4aa7a3a8c6bd45dfebf0` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:24.072019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f053d44-3090-49bb-8f4d-41aae8031104 None admin] RBAC: Authorizing `identity:get_user(user_id=79c76db5606f4aa7a3a8c6bd45dfebf0)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:24.073701 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f053d44-3090-49bb-8f4d-41aae8031104 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:24.075688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 134/266] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:23 2022] GET /identity/v3/users/79c76db5606f4aa7a3a8c6bd45dfebf0 => generated 283 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:24.082115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c732b1-8b3e-4847-97c7-8c6fee3ce9b0 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:24.100461 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:24.205228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c732b1-8b3e-4847-97c7-8c6fee3ce9b0 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:24.239445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbcff1-8d67-42d6-8011-c36e17b5a45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:24.240104 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbcff1-8d67-42d6-8011-c36e17b5a45e None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:24.240393 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbcff1-8d67-42d6-8011-c36e17b5a45e None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:24.240669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbcff1-8d67-42d6-8011-c36e17b5a45e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:24.311170 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbcff1-8d67-42d6-8011-c36e17b5a45e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:24.312785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbcff1-8d67-42d6-8011-c36e17b5a45e None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:24.324747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:24.423205 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f053d44-3090-49bb-8f4d-41aae8031104 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:24.424343 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 133/267] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:24.465539 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:24.572843 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f053d44-3090-49bb-8f4d-41aae8031104 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:24.606951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c1ed8-d821-4126-b127-0c5338e9b4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:24.607867 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c1ed8-d821-4126-b127-0c5338e9b4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:24.608159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c1ed8-d821-4126-b127-0c5338e9b4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:24.608976 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c1ed8-d821-4126-b127-0c5338e9b4f2 None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:24.610171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c1ed8-d821-4126-b127-0c5338e9b4f2 None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:24.611710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c1ed8-d821-4126-b127-0c5338e9b4f2 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:24.613735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 135/268] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:24 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:24.619270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbcff1-8d67-42d6-8011-c36e17b5a45e None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:24.777510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbcff1-8d67-42d6-8011-c36e17b5a45e None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:24.797064 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:24.828584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92f999-0110-4462-b93b-5d6631de7235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:24.829259 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92f999-0110-4462-b93b-5d6631de7235 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:24.829541 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92f999-0110-4462-b93b-5d6631de7235 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:24.829817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92f999-0110-4462-b93b-5d6631de7235 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:24.830660 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92f999-0110-4462-b93b-5d6631de7235 None admin] RBAC: Adding query filter params (role.id=05ca4843c34e463c94de09f756d95318, scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7, user.id=79c76db5606f4aa7a3a8c6bd45dfebf0) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:24.830995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92f999-0110-4462-b93b-5d6631de7235 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:24.832437 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92f999-0110-4462-b93b-5d6631de7235 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:24.844036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 134/269] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:24 2022] GET /identity/v3/role_assignments?user.id=79c76db5606f4aa7a3a8c6bd45dfebf0&scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7&role.id=05ca4843c34e463c94de09f756d95318 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:24.977727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a942bb-9277-4314-88f9-6e2de2ac393e None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:24.978191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a942bb-9277-4314-88f9-6e2de2ac393e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:24.978575 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a942bb-9277-4314-88f9-6e2de2ac393e None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:24.980151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 136/270] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:24.986792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324a816-cd22-436a-b41f-79768f4a100e None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:24.987309 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324a816-cd22-436a-b41f-79768f4a100e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:24.987590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324a816-cd22-436a-b41f-79768f4a100e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:25.005903 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-b324a816-cd22-436a-b41f-79768f4a100e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:25.007148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b324a816-cd22-436a-b41f-79768f4a100e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:25.087758 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 135/271] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:25.111722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a942bb-9277-4314-88f9-6e2de2ac393e None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:25.166379 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62538813-00c8-4d9a-9c7b-f88a6a99fc58 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:25.166700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62538813-00c8-4d9a-9c7b-f88a6a99fc58 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:25.166968 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62538813-00c8-4d9a-9c7b-f88a6a99fc58 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:25.168354 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 136/272] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:59:25.174067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5abc6b-190f-4812-b935-724908a29888 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:25.174362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5abc6b-190f-4812-b935-724908a29888 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:25.174641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5abc6b-190f-4812-b935-724908a29888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:25.193195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-ff5abc6b-190f-4812-b935-724908a29888 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:25.194610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ff5abc6b-190f-4812-b935-724908a29888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:25.223057 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:25.239139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a942bb-9277-4314-88f9-6e2de2ac393e None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:25.287371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d17e7-617d-4809-94e0-2e5b63266147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:25.288288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d17e7-617d-4809-94e0-2e5b63266147 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:25.288684 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d17e7-617d-4809-94e0-2e5b63266147 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:25.288971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d17e7-617d-4809-94e0-2e5b63266147 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:25.291489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 137/273] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:25.300222 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d17e7-617d-4809-94e0-2e5b63266147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:25.308017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-106d17e7-617d-4809-94e0-2e5b63266147 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:25.308017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:25.308017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:25.308017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:25.308017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:25.308017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:25.308017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:25.308017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:25.308017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:25.308017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:25.308017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:25.308017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:25.308017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:25.308017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:25.308017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:25.308017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:25.308898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:25.308898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:25.308898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:25.308898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:25.308898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:25.308898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:25.308898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:25.308898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:25.308898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:25.308898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:25.308898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:25.308898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:25.308898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:25.308898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:25.308898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:25.308898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:25.309695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:25.309695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:25.309695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:25.309695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:25.309695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:25.309695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:25.309695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:25.309695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:25.309695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:25.309695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:25.309695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:25.309695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:25.309695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 137/274] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:25.316402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5abc6b-190f-4812-b935-724908a29888 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:25.326529 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d17e7-617d-4809-94e0-2e5b63266147 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:25.360463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:25.441952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5abc6b-190f-4812-b935-724908a29888 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:25.453121 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d17e7-617d-4809-94e0-2e5b63266147 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:25.478271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890a1ae-0d35-4bf9-8376-4645e9bef7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:25.479242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890a1ae-0d35-4bf9-8376-4645e9bef7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:25.479242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890a1ae-0d35-4bf9-8376-4645e9bef7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:25.479532 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890a1ae-0d35-4bf9-8376-4645e9bef7b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:25.480295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890a1ae-0d35-4bf9-8376-4645e9bef7b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:25.480590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890a1ae-0d35-4bf9-8376-4645e9bef7b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:25.481981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890a1ae-0d35-4bf9-8376-4645e9bef7b3 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:25.491235 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da15f0-f168-44fe-a653-1909fd029870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:25.491800 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 138/275] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:25.492866 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da15f0-f168-44fe-a653-1909fd029870 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:25.493368 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da15f0-f168-44fe-a653-1909fd029870 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:25.493844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da15f0-f168-44fe-a653-1909fd029870 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:25.495453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da15f0-f168-44fe-a653-1909fd029870 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:25.497127 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890a1ae-0d35-4bf9-8376-4645e9bef7b3 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:25.497616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da15f0-f168-44fe-a653-1909fd029870 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:25.505113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 138/276] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:25.506234 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da15f0-f168-44fe-a653-1909fd029870 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:25.528407 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:25.602814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:25.608617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890a1ae-0d35-4bf9-8376-4645e9bef7b3 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:25.628176 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da15f0-f168-44fe-a653-1909fd029870 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:25.644107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b64fa-54e3-4e00-b301-e9c03675283f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:25.644992 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b64fa-54e3-4e00-b301-e9c03675283f None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:25.645275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b64fa-54e3-4e00-b301-e9c03675283f None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:25.645623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b64fa-54e3-4e00-b301-e9c03675283f None admin] PATH_INFO: `/v3/users/e5584d0226e943469166fe4d3764b3d5` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:25.660048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b64fa-54e3-4e00-b301-e9c03675283f None admin] RBAC: Authorizing `identity:get_user(user_id=e5584d0226e943469166fe4d3764b3d5)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:25.661534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b64fa-54e3-4e00-b301-e9c03675283f None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:25.666758 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 139/277] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:25 2022] GET /identity/v3/users/e5584d0226e943469166fe4d3764b3d5 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:25.669287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b64fa-54e3-4e00-b301-e9c03675283f None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:25.672403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cac0dc-a33e-4cba-bb67-4aacd1c9a417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:25.673163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cac0dc-a33e-4cba-bb67-4aacd1c9a417 None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:25.673568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cac0dc-a33e-4cba-bb67-4aacd1c9a417 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:25.673860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cac0dc-a33e-4cba-bb67-4aacd1c9a417 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:25.674593 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cac0dc-a33e-4cba-bb67-4aacd1c9a417 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:25.677282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cac0dc-a33e-4cba-bb67-4aacd1c9a417 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:25.706802 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 139/278] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 08:59:25 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:59:25.788745 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b64fa-54e3-4e00-b301-e9c03675283f None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:25.829298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2026d6-7daa-46df-8acf-d4833fa2354f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:25.830382 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2026d6-7daa-46df-8acf-d4833fa2354f None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:25.830985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2026d6-7daa-46df-8acf-d4833fa2354f None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:25.831478 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2026d6-7daa-46df-8acf-d4833fa2354f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:25.832545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2026d6-7daa-46df-8acf-d4833fa2354f None admin] RBAC: Adding query filter params (role.id=0e2011aac3e1430cb96bdbf810e89eb7, user.id=e5584d0226e943469166fe4d3764b3d5, scope.system=all) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:25.833067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2026d6-7daa-46df-8acf-d4833fa2354f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:25.834783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2026d6-7daa-46df-8acf-d4833fa2354f None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:25.841349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 140/279] 10.222.0.44 () {56 vars in 1308 bytes} [Wed Jul 20 08:59:25 2022] GET /identity/v3/role_assignments?user.id=e5584d0226e943469166fe4d3764b3d5&scope.system=all&role.id=0e2011aac3e1430cb96bdbf810e89eb7 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:25.919793 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613a0e1-3539-4347-9894-a0c6ad2a43a2 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:25.920082 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613a0e1-3539-4347-9894-a0c6ad2a43a2 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:25.920313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613a0e1-3539-4347-9894-a0c6ad2a43a2 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:25.922117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 140/280] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:25.928601 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7695e1c-a740-49fc-9c71-e1de4b37db81 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:25.929184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7695e1c-a740-49fc-9c71-e1de4b37db81 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:25.929646 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7695e1c-a740-49fc-9c71-e1de4b37db81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:25.948938 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a7695e1c-a740-49fc-9c71-e1de4b37db81 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:25.950149 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a7695e1c-a740-49fc-9c71-e1de4b37db81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:25.967134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:26.004053 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76bd15-e33b-4e4b-b07e-d573da9fb293 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:26.004694 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76bd15-e33b-4e4b-b07e-d573da9fb293 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:26.005159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76bd15-e33b-4e4b-b07e-d573da9fb293 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:26.010213 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 141/281] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:26.014217 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae35aab-b034-42ad-ad41-b132c0ede45e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:26.014345 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae35aab-b034-42ad-ad41-b132c0ede45e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:26.015403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae35aab-b034-42ad-ad41-b132c0ede45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:26.041123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-cae35aab-b034-42ad-ad41-b132c0ede45e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:26.042431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-cae35aab-b034-42ad-ad41-b132c0ede45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:26.045755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 141/282] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:26.070648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7695e1c-a740-49fc-9c71-e1de4b37db81 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:26.124992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 142/283] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:26.147841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae35aab-b034-42ad-ad41-b132c0ede45e None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:26.151844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:26.195257 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7695e1c-a740-49fc-9c71-e1de4b37db81 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:26.230809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c6cd7-9255-4cdb-b66b-10932739aa6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:26.231964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c6cd7-9255-4cdb-b66b-10932739aa6c None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:26.232453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c6cd7-9255-4cdb-b66b-10932739aa6c None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:26.232935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c6cd7-9255-4cdb-b66b-10932739aa6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:26.244603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49c6cd7-9255-4cdb-b66b-10932739aa6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:26.253276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-a49c6cd7-9255-4cdb-b66b-10932739aa6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:26.253276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:26.253276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:26.253276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:26.253276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:26.253276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:26.253276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:26.253276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:26.253276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:26.253276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:26.253276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:26.253276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:26.253276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:26.253276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:26.253276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:26.253276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:26.254132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:26.254132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:26.254132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:26.254132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:26.254132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:26.254132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:26.254132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:26.254132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:26.254132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:26.254132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:26.254132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:26.254132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:26.254132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:26.254132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:26.254132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:26.254132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:26.255824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:26.255824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:26.255824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:26.255824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:26.255824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:26.255824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:26.255824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:26.255824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:26.255824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:26.255824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:26.255824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:26.255824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:26.257645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae35aab-b034-42ad-ad41-b132c0ede45e None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:26.257911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 142/284] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:26.263553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c6cd7-9255-4cdb-b66b-10932739aa6c None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:26.304309 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:26.377559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c6cd7-9255-4cdb-b66b-10932739aa6c None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:26.380724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c52a3c-497e-40b1-aa82-72e84de9125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:26.381545 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c52a3c-497e-40b1-aa82-72e84de9125e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:26.381831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c52a3c-497e-40b1-aa82-72e84de9125e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:26.382107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c52a3c-497e-40b1-aa82-72e84de9125e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:26.418122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc9d09-10f9-4551-9620-c079069feeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:26.418951 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc9d09-10f9-4551-9620-c079069feeec None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:26.419310 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc9d09-10f9-4551-9620-c079069feeec None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:26.419705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc9d09-10f9-4551-9620-c079069feeec None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:26.420941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc9d09-10f9-4551-9620-c079069feeec None admin] RBAC: Adding query filter params (name=member) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:26.422081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc9d09-10f9-4551-9620-c079069feeec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:26.423777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc9d09-10f9-4551-9620-c079069feeec None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:26.434626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 143/285] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:26.435326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c52a3c-497e-40b1-aa82-72e84de9125e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:26.440611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc9d09-10f9-4551-9620-c079069feeec None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:26.443374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-b8c52a3c-497e-40b1-aa82-72e84de9125e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:26.443374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:26.443374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:26.443374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:26.443374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:26.443374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:26.443374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:26.443374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:26.443374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:26.443374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:26.443374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:26.443374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:26.443374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:26.443374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:26.443374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:26.443374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:26.444535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:26.444535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:26.444535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:26.444535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:26.444535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:26.444535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:26.444535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:26.444535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:26.444535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:26.444535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:26.444535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:26.444535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:26.444535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:26.444535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:26.444535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:26.444535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:26.445372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:26.445372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:26.445372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:26.445372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:26.445372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:26.445372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:26.445372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:26.445372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:26.445372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:26.445372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:26.445372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:26.445372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:26.453401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 143/286] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:26.460785 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c52a3c-497e-40b1-aa82-72e84de9125e None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:26.517023 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:26.557339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc9d09-10f9-4551-9620-c079069feeec None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:26.579068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:26.594601 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabd902-b3e3-4329-8f14-4537bdfd332a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:26.595879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabd902-b3e3-4329-8f14-4537bdfd332a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:26.596380 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabd902-b3e3-4329-8f14-4537bdfd332a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:26.596852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabd902-b3e3-4329-8f14-4537bdfd332a None admin] PATH_INFO: `/v3/users/eec1890ca3ac431e8737395b1e8bc927` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:26.598266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabd902-b3e3-4329-8f14-4537bdfd332a None admin] RBAC: Authorizing `identity:get_user(user_id=eec1890ca3ac431e8737395b1e8bc927)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:26.600008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabd902-b3e3-4329-8f14-4537bdfd332a None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:26.606426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 144/287] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:26 2022] GET /identity/v3/users/eec1890ca3ac431e8737395b1e8bc927 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:26.607895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabd902-b3e3-4329-8f14-4537bdfd332a None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:26.657190 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c52a3c-497e-40b1-aa82-72e84de9125e None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:26.712152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabd902-b3e3-4329-8f14-4537bdfd332a None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:26.735919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319baa8-c864-46ed-ab60-1f17a0f7324c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:26.736604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319baa8-c864-46ed-ab60-1f17a0f7324c None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:26.737037 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319baa8-c864-46ed-ab60-1f17a0f7324c None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:26.737270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319baa8-c864-46ed-ab60-1f17a0f7324c None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:26.738042 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319baa8-c864-46ed-ab60-1f17a0f7324c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:26.738614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319baa8-c864-46ed-ab60-1f17a0f7324c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:26.739888 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319baa8-c864-46ed-ab60-1f17a0f7324c None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:26.752966 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d224d-4125-4959-8987-d6d35cfff8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:26.754835 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d224d-4125-4959-8987-d6d35cfff8fd None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:26.755477 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d224d-4125-4959-8987-d6d35cfff8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:26.757498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 144/288] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:26.758050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d224d-4125-4959-8987-d6d35cfff8fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:26.763937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319baa8-c864-46ed-ab60-1f17a0f7324c None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:26.779036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:26.842036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d224d-4125-4959-8987-d6d35cfff8fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:26.843925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d224d-4125-4959-8987-d6d35cfff8fd None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:26.949414 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 145/289] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:26.951065 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d224d-4125-4959-8987-d6d35cfff8fd None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:26.999186 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:27.069792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319baa8-c864-46ed-ab60-1f17a0f7324c None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:27.097869 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:27.104376 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d224d-4125-4959-8987-d6d35cfff8fd None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:27.107907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8428fc-77e1-4bc2-aad4-0dee2c65a6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:27.109108 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8428fc-77e1-4bc2-aad4-0dee2c65a6ff None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:27.109505 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8428fc-77e1-4bc2-aad4-0dee2c65a6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:27.109784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8428fc-77e1-4bc2-aad4-0dee2c65a6ff None admin] PATH_INFO: `/v3/users/00e0957363bd4c38a7bba6d78abd168e` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:27.111034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8428fc-77e1-4bc2-aad4-0dee2c65a6ff None admin] RBAC: Authorizing `identity:get_user(user_id=00e0957363bd4c38a7bba6d78abd168e)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:27.113526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8428fc-77e1-4bc2-aad4-0dee2c65a6ff None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:27.115615 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 145/290] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:26 2022] GET /identity/v3/users/00e0957363bd4c38a7bba6d78abd168e => generated 297 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:27.121358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8428fc-77e1-4bc2-aad4-0dee2c65a6ff None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:27.184782 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfe4d4-da86-4585-ac60-bbd32d72be4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:27.186352 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfe4d4-da86-4585-ac60-bbd32d72be4e None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:27.187228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfe4d4-da86-4585-ac60-bbd32d72be4e None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:27.187515 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfe4d4-da86-4585-ac60-bbd32d72be4e None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:27.192477 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecfe4d4-da86-4585-ac60-bbd32d72be4e None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:27.194247 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecfe4d4-da86-4585-ac60-bbd32d72be4e None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:27.201358 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 146/291] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:26 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:27.202987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfe4d4-da86-4585-ac60-bbd32d72be4e None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:27.315060 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfe4d4-da86-4585-ac60-bbd32d72be4e None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:27.323631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:27.341502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8428fc-77e1-4bc2-aad4-0dee2c65a6ff None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:27.354155 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995d0a3-3828-440c-8de8-4810121848a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:27.354842 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995d0a3-3828-440c-8de8-4810121848a9 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:27.355126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995d0a3-3828-440c-8de8-4810121848a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:27.355403 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995d0a3-3828-440c-8de8-4810121848a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:27.356234 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995d0a3-3828-440c-8de8-4810121848a9 None admin] RBAC: Adding query filter params (role.id=0e2011aac3e1430cb96bdbf810e89eb7, scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7, user.id=eec1890ca3ac431e8737395b1e8bc927) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:27.356546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995d0a3-3828-440c-8de8-4810121848a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:27.362711 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995d0a3-3828-440c-8de8-4810121848a9 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:27.370907 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 147/292] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:27 2022] GET /identity/v3/role_assignments?user.id=eec1890ca3ac431e8737395b1e8bc927&scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7&role.id=0e2011aac3e1430cb96bdbf810e89eb7 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:27.425044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e68d7-1b40-4a2e-9099-4cd30869ba7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:27.426330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e68d7-1b40-4a2e-9099-4cd30869ba7d None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:27.426855 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e68d7-1b40-4a2e-9099-4cd30869ba7d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:27.427341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e68d7-1b40-4a2e-9099-4cd30869ba7d None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:27.431458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e68d7-1b40-4a2e-9099-4cd30869ba7d None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:27.433226 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e68d7-1b40-4a2e-9099-4cd30869ba7d None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:27.435325 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 146/293] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:27 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:27.441514 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995d0a3-3828-440c-8de8-4810121848a9 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:27.588933 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995d0a3-3828-440c-8de8-4810121848a9 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:27.626171 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84adb2db-7316-4973-b858-e142feee82ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:27.627166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adb2db-7316-4973-b858-e142feee82ac None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:27.627452 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adb2db-7316-4973-b858-e142feee82ac None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:27.627733 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adb2db-7316-4973-b858-e142feee82ac None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7/users/00e0957363bd4c38a7bba6d78abd168e/roles/559659e5dbc346d79735375b9130736d` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:27.629817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adb2db-7316-4973-b858-e142feee82ac None admin] RBAC: Authorizing `identity:create_grant(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7, user_id=00e0957363bd4c38a7bba6d78abd168e, role_id=559659e5dbc346d79735375b9130736d)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:27.631323 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adb2db-7316-4973-b858-e142feee82ac None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:27.650528 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 148/294] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 08:59:27 2022] PUT /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7/users/00e0957363bd4c38a7bba6d78abd168e/roles/559659e5dbc346d79735375b9130736d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:59:28.227731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc687645-0fb0-4452-adcd-49e559e480f7 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:28.228104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc687645-0fb0-4452-adcd-49e559e480f7 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:28.228298 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc687645-0fb0-4452-adcd-49e559e480f7 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:28.229704 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 147/295] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:28.232443 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb3090-4604-4c86-b627-2b5717040a16 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:28.232888 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb3090-4604-4c86-b627-2b5717040a16 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:28.233025 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb3090-4604-4c86-b627-2b5717040a16 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:28.235931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e77c9-e954-4e2b-a8e4-3c8996debe04 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:28.236221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e77c9-e954-4e2b-a8e4-3c8996debe04 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:28.236503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e77c9-e954-4e2b-a8e4-3c8996debe04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:28.238279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 149/296] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:28.240307 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f62549-f491-49f1-87d8-ee40f3cc5d6f None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:28.240834 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f62549-f491-49f1-87d8-ee40f3cc5d6f None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:28.241315 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f62549-f491-49f1-87d8-ee40f3cc5d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:28.255638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-b25e77c9-e954-4e2b-a8e4-3c8996debe04 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:28.257162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b25e77c9-e954-4e2b-a8e4-3c8996debe04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:28.258644 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-28f62549-f491-49f1-87d8-ee40f3cc5d6f None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:28.260328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-28f62549-f491-49f1-87d8-ee40f3cc5d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:28.288463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:28.335982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:28.343981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 150/297] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:28.351358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 148/298] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:28.370304 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f62549-f491-49f1-87d8-ee40f3cc5d6f None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:28.376044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e77c9-e954-4e2b-a8e4-3c8996debe04 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:28.484223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f62549-f491-49f1-87d8-ee40f3cc5d6f None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:28.484397 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e77c9-e954-4e2b-a8e4-3c8996debe04 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:28.521696 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c49e7b-c24d-4fc3-9bda-55cfa3796e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:28.521875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2245f-b778-4117-801b-5bea13b2f210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:28.522497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c49e7b-c24d-4fc3-9bda-55cfa3796e18 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:28.522598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2245f-b778-4117-801b-5bea13b2f210 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:28.522813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c49e7b-c24d-4fc3-9bda-55cfa3796e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:28.522910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2245f-b778-4117-801b-5bea13b2f210 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:28.523094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2245f-b778-4117-801b-5bea13b2f210 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:28.523205 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c49e7b-c24d-4fc3-9bda-55cfa3796e18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:28.524331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c49e7b-c24d-4fc3-9bda-55cfa3796e18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:28.525799 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c49e7b-c24d-4fc3-9bda-55cfa3796e18 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:28.527455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 149/299] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:28.528651 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90324f31-aa43-49af-8c90-00e12068af93 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:28.528940 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90324f31-aa43-49af-8c90-00e12068af93 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:28.529219 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90324f31-aa43-49af-8c90-00e12068af93 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:28.530717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 150/300] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:28.537665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90324f31-aa43-49af-8c90-00e12068af93 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:28.541749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c2245f-b778-4117-801b-5bea13b2f210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:28.549840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-32c2245f-b778-4117-801b-5bea13b2f210 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:28.549840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:28.549840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:28.549840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:28.549840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:28.549840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:28.549840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:28.549840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:28.549840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:28.549840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:28.549840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:28.549840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:28.549840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:28.549840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:28.549840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:28.549840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:28.550941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:28.550941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:28.550941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:28.550941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:28.550941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:28.550941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:28.550941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:28.550941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:28.550941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:28.550941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:28.550941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:28.550941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:28.550941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:28.550941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:28.550941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:28.550941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:28.551772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:28.551772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:28.551772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:28.551772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:28.551772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:28.551772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:28.551772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:28.551772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:28.551772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:28.551772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:28.551772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:28.551772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:28.555971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 151/301] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:28.557303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85d638-a68d-4cb1-b079-fb1dac72a350 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:28.557505 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85d638-a68d-4cb1-b079-fb1dac72a350 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:28.557789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85d638-a68d-4cb1-b079-fb1dac72a350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:28.575399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-db85d638-a68d-4cb1-b079-fb1dac72a350 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:28.577294 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-db85d638-a68d-4cb1-b079-fb1dac72a350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:28.592539 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:28.606742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:28.651231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 152/302] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:28.652492 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85d638-a68d-4cb1-b079-fb1dac72a350 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:28.659378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90324f31-aa43-49af-8c90-00e12068af93 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:28.697485 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab2a6b-6dd4-4ce1-8bb2-1d215bae9ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:28.698156 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab2a6b-6dd4-4ce1-8bb2-1d215bae9ec2 None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:28.698442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab2a6b-6dd4-4ce1-8bb2-1d215bae9ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:28.698755 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab2a6b-6dd4-4ce1-8bb2-1d215bae9ec2 None admin] PATH_INFO: `/v3/groups` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:28.699456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab2a6b-6dd4-4ce1-8bb2-1d215bae9ec2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:28.700889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab2a6b-6dd4-4ce1-8bb2-1d215bae9ec2 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:28.716675 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 151/303] 10.222.0.44 () {60 vars in 1134 bytes} [Wed Jul 20 08:59:28 2022] POST /identity/v3/groups => generated 229 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:59:28.718258 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab2a6b-6dd4-4ce1-8bb2-1d215bae9ec2 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:28.791863 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85d638-a68d-4cb1-b079-fb1dac72a350 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:28.836526 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b5963-3f36-4a10-b264-c1f44d404930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:28.837637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b5963-3f36-4a10-b264-c1f44d404930 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:28.838137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b5963-3f36-4a10-b264-c1f44d404930 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:28.838704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b5963-3f36-4a10-b264-c1f44d404930 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:28.839744 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b5963-3f36-4a10-b264-c1f44d404930 None admin] RBAC: Adding query filter params (name=member) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:28.840363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b5963-3f36-4a10-b264-c1f44d404930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:28.841981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b5963-3f36-4a10-b264-c1f44d404930 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:28.851891 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 153/304] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:28.857631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b5963-3f36-4a10-b264-c1f44d404930 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:28.859656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab2a6b-6dd4-4ce1-8bb2-1d215bae9ec2 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:28.876678 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:28.885477 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:28.902012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f1e18-77e8-4454-8749-346277670b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:28.903682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f1e18-77e8-4454-8749-346277670b89 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:28.903971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f1e18-77e8-4454-8749-346277670b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:28.904251 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f1e18-77e8-4454-8749-346277670b89 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:28.919115 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f1e18-77e8-4454-8749-346277670b89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:28.929318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-f94f1e18-77e8-4454-8749-346277670b89 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:28.929318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:28.929318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:28.929318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:28.929318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:28.929318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:28.929318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:28.929318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:28.929318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:28.929318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:28.929318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:28.929318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:28.929318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:28.929318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:28.929318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:28.929318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:28.930225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:28.930225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:28.930225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:28.930225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:28.930225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:28.930225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:28.930225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:28.930225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:28.930225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:28.930225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:28.930225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:28.930225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:28.930225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:28.930225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:28.930225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:28.930225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:28.931487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:28.931487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:28.931487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:28.931487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:28.931487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:28.931487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:28.931487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:28.931487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:28.931487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:28.931487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:28.931487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:28.931487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:28.931487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 152/305] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:28.936285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f1e18-77e8-4454-8749-346277670b89 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:28.990035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b5963-3f36-4a10-b264-c1f44d404930 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:29.040184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc1bd8-15c4-4abc-ad6c-2906b1049977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:29.041633 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc1bd8-15c4-4abc-ad6c-2906b1049977 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:29.042424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc1bd8-15c4-4abc-ad6c-2906b1049977 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:29.043295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc1bd8-15c4-4abc-ad6c-2906b1049977 None admin] PATH_INFO: `/v3/users/e5584d0226e943469166fe4d3764b3d5` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:29.044791 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc1bd8-15c4-4abc-ad6c-2906b1049977 None admin] RBAC: Authorizing `identity:get_user(user_id=e5584d0226e943469166fe4d3764b3d5)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:29.047562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc1bd8-15c4-4abc-ad6c-2906b1049977 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:29.049391 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 154/306] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:28 2022] GET /identity/v3/users/e5584d0226e943469166fe4d3764b3d5 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:29.055694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc1bd8-15c4-4abc-ad6c-2906b1049977 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:29.063954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f1e18-77e8-4454-8749-346277670b89 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:29.111311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24cd60-2784-47ba-9b6b-c71609c32582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:29.112864 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24cd60-2784-47ba-9b6b-c71609c32582 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:29.113633 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24cd60-2784-47ba-9b6b-c71609c32582 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:29.114142 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24cd60-2784-47ba-9b6b-c71609c32582 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:29.115425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24cd60-2784-47ba-9b6b-c71609c32582 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:29.116340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24cd60-2784-47ba-9b6b-c71609c32582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:29.118414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24cd60-2784-47ba-9b6b-c71609c32582 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:29.130868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 153/307] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:29.135314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24cd60-2784-47ba-9b6b-c71609c32582 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:29.146877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:29.169644 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc1bd8-15c4-4abc-ad6c-2906b1049977 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:29.192241 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:29.209041 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5930c5-cdc5-42eb-a38b-4251fb910222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:29.210495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5930c5-cdc5-42eb-a38b-4251fb910222 None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:29.211048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5930c5-cdc5-42eb-a38b-4251fb910222 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:29.211577 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5930c5-cdc5-42eb-a38b-4251fb910222 None admin] PATH_INFO: `/v3/system/users/e5584d0226e943469166fe4d3764b3d5/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:29.213488 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5930c5-cdc5-42eb-a38b-4251fb910222 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e5584d0226e943469166fe4d3764b3d5, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:29.215117 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5930c5-cdc5-42eb-a38b-4251fb910222 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:29.234729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 155/308] 10.222.0.44 () {58 vars in 1334 bytes} [Wed Jul 20 08:59:29 2022] PUT /identity/v3/system/users/e5584d0226e943469166fe4d3764b3d5/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:59:29.262195 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24cd60-2784-47ba-9b6b-c71609c32582 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:29.301139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b014207-3555-43b0-9382-4ca63a3abc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:29.301944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b014207-3555-43b0-9382-4ca63a3abc74 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:29.302225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b014207-3555-43b0-9382-4ca63a3abc74 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:29.302498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b014207-3555-43b0-9382-4ca63a3abc74 None admin] PATH_INFO: `/v3/users/c62cb7d9924c41dbac763b2f862c6285` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:29.319599 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b014207-3555-43b0-9382-4ca63a3abc74 None admin] RBAC: Authorizing `identity:get_user(user_id=c62cb7d9924c41dbac763b2f862c6285)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:29.321207 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b014207-3555-43b0-9382-4ca63a3abc74 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:29.323119 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 154/309] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:29 2022] GET /identity/v3/users/c62cb7d9924c41dbac763b2f862c6285 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:29.329395 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5930c5-cdc5-42eb-a38b-4251fb910222 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:29.444956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5930c5-cdc5-42eb-a38b-4251fb910222 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:29.481071 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af61c90-733c-4cc8-9724-620fa080ce89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:29.481737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af61c90-733c-4cc8-9724-620fa080ce89 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:29.482023 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af61c90-733c-4cc8-9724-620fa080ce89 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:29.482304 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af61c90-733c-4cc8-9724-620fa080ce89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:29.483187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af61c90-733c-4cc8-9724-620fa080ce89 None admin] RBAC: Adding query filter params (role.id=559659e5dbc346d79735375b9130736d, user.id=c62cb7d9924c41dbac763b2f862c6285, scope.system=all) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:29.483584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af61c90-733c-4cc8-9724-620fa080ce89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:29.485003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af61c90-733c-4cc8-9724-620fa080ce89 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:29.490482 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 156/310] 10.222.0.44 () {56 vars in 1308 bytes} [Wed Jul 20 08:59:29 2022] GET /identity/v3/role_assignments?user.id=c62cb7d9924c41dbac763b2f862c6285&scope.system=all&role.id=559659e5dbc346d79735375b9130736d => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:29.797743 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bc0cf-6876-4099-a132-774d2ad92233 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:29.798277 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bc0cf-6876-4099-a132-774d2ad92233 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:29.798277 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bc0cf-6876-4099-a132-774d2ad92233 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:29.799994 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 155/311] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:29.805282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508cb6c-a63f-4835-a3c7-38fa64066519 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:29.805520 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508cb6c-a63f-4835-a3c7-38fa64066519 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:29.805811 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508cb6c-a63f-4835-a3c7-38fa64066519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:29.823155 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-3508cb6c-a63f-4835-a3c7-38fa64066519 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:29.824344 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3508cb6c-a63f-4835-a3c7-38fa64066519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:29.826362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:29.896436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 157/312] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:29.919430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bc0cf-6876-4099-a132-774d2ad92233 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:30.026365 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:30.030888 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bc0cf-6876-4099-a132-774d2ad92233 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:30.066897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21108869-0883-42b6-a165-62de6e4d1f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:30.067622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21108869-0883-42b6-a165-62de6e4d1f88 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:30.067906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21108869-0883-42b6-a165-62de6e4d1f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:30.068185 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21108869-0883-42b6-a165-62de6e4d1f88 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:30.079470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21108869-0883-42b6-a165-62de6e4d1f88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:30.087147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-21108869-0883-42b6-a165-62de6e4d1f88 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:30.087147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:30.087147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:30.087147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:30.087147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:30.087147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:30.087147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:30.087147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:30.087147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:30.087147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:30.087147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:30.087147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:30.087147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:30.087147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:30.087147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:30.087147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:30.088024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:30.088024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:30.088024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:30.088024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:30.088024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:30.088024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:30.088024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:30.088024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:30.088024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:30.088024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:30.088024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:30.088024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:30.088024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:30.088024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:30.088024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:30.088024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:30.088806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:30.088806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:30.088806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:30.088806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:30.088806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:30.088806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:30.088806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:30.088806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:30.088806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:30.088806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:30.088806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:30.088806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:30.088806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 156/313] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:30.094901 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508cb6c-a63f-4835-a3c7-38fa64066519 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:30.198311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508cb6c-a63f-4835-a3c7-38fa64066519 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:30.236052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e6d3f-2fd0-4242-bec8-e4774716a4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:30.236800 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e6d3f-2fd0-4242-bec8-e4774716a4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:30.236998 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e6d3f-2fd0-4242-bec8-e4774716a4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:30.237275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e6d3f-2fd0-4242-bec8-e4774716a4f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:30.238028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e6d3f-2fd0-4242-bec8-e4774716a4f3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:30.238333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e6d3f-2fd0-4242-bec8-e4774716a4f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:30.239737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e6d3f-2fd0-4242-bec8-e4774716a4f3 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:30.244875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:30.248566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 158/314] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:30.254902 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21108869-0883-42b6-a165-62de6e4d1f88 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:30.358173 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21108869-0883-42b6-a165-62de6e4d1f88 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:30.393527 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4f6d1-70e1-4f04-bdea-0ebef473ef03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:30.394711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4f6d1-70e1-4f04-bdea-0ebef473ef03 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:30.395201 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4f6d1-70e1-4f04-bdea-0ebef473ef03 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:30.395654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4f6d1-70e1-4f04-bdea-0ebef473ef03 None admin] PATH_INFO: `/v3/users/00e0957363bd4c38a7bba6d78abd168e` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:30.397020 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4f6d1-70e1-4f04-bdea-0ebef473ef03 None admin] RBAC: Authorizing `identity:get_user(user_id=00e0957363bd4c38a7bba6d78abd168e)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:30.398640 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4f6d1-70e1-4f04-bdea-0ebef473ef03 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:30.400763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 157/315] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:30 2022] GET /identity/v3/users/00e0957363bd4c38a7bba6d78abd168e => generated 297 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:30.407800 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e6d3f-2fd0-4242-bec8-e4774716a4f3 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:30.522438 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e6d3f-2fd0-4242-bec8-e4774716a4f3 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:30.558523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d15df-58e4-4c23-a857-163549923767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:30.559212 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d15df-58e4-4c23-a857-163549923767 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:30.559496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d15df-58e4-4c23-a857-163549923767 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:30.559772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d15df-58e4-4c23-a857-163549923767 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:30.630728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:30.633981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d15df-58e4-4c23-a857-163549923767 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:30.635730 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d15df-58e4-4c23-a857-163549923767 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:30.738860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 159/316] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:30.744294 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4f6d1-70e1-4f04-bdea-0ebef473ef03 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:30.773319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:30.848195 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4f6d1-70e1-4f04-bdea-0ebef473ef03 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:30.888231 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e2717-3cb3-4329-a15f-be06bba805d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:30.889022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e2717-3cb3-4329-a15f-be06bba805d2 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:30.889305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e2717-3cb3-4329-a15f-be06bba805d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:30.890184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e2717-3cb3-4329-a15f-be06bba805d2 None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:30.891411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e2717-3cb3-4329-a15f-be06bba805d2 None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:30.892847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e2717-3cb3-4329-a15f-be06bba805d2 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:30.894862 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 158/317] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:30 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:30.900556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d15df-58e4-4c23-a857-163549923767 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:30.965874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805b4a6-c5ff-49da-bfc5-8b5c7b38cdf4 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:30.966197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805b4a6-c5ff-49da-bfc5-8b5c7b38cdf4 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:30.966435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805b4a6-c5ff-49da-bfc5-8b5c7b38cdf4 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:30.968726 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 159/318] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:59:30.973851 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d9bf7-70c4-423d-b8bf-a123d2e32794 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:30.974479 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d9bf7-70c4-423d-b8bf-a123d2e32794 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:30.975134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d9bf7-70c4-423d-b8bf-a123d2e32794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:30.993463 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-ae1d9bf7-70c4-423d-b8bf-a123d2e32794 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:30.994891 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ae1d9bf7-70c4-423d-b8bf-a123d2e32794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:31.006572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:31.010603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d15df-58e4-4c23-a857-163549923767 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:31.049914 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde66cf3-25f0-4e27-8d50-400bea8371ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:31.050579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde66cf3-25f0-4e27-8d50-400bea8371ed None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:31.050890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde66cf3-25f0-4e27-8d50-400bea8371ed None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:31.051169 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde66cf3-25f0-4e27-8d50-400bea8371ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:31.052008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde66cf3-25f0-4e27-8d50-400bea8371ed None admin] RBAC: Adding query filter params (role.id=559659e5dbc346d79735375b9130736d, scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7, user.id=00e0957363bd4c38a7bba6d78abd168e) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:31.052321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde66cf3-25f0-4e27-8d50-400bea8371ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:31.053760 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde66cf3-25f0-4e27-8d50-400bea8371ed None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:31.060442 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 160/319] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:30 2022] GET /identity/v3/role_assignments?user.id=00e0957363bd4c38a7bba6d78abd168e&scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7&role.id=559659e5dbc346d79735375b9130736d => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:31.076396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 160/320] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:31.100514 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde66cf3-25f0-4e27-8d50-400bea8371ed None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:31.216341 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde66cf3-25f0-4e27-8d50-400bea8371ed None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:31.261001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2bd675-0774-4c8b-afb9-ed8a607b7212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:31.262087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bd675-0774-4c8b-afb9-ed8a607b7212 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:31.262374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bd675-0774-4c8b-afb9-ed8a607b7212 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:31.262648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bd675-0774-4c8b-afb9-ed8a607b7212 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:31.263864 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2bd675-0774-4c8b-afb9-ed8a607b7212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:31.266445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2bd675-0774-4c8b-afb9-ed8a607b7212 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:31.268101 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 161/321] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:31.273750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d9bf7-70c4-423d-b8bf-a123d2e32794 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:31.302942 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:31.386774 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d9bf7-70c4-423d-b8bf-a123d2e32794 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:31.388855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d34600-01df-46e1-bd70-d2ebedb92fe1 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:31.389153 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d34600-01df-46e1-bd70-d2ebedb92fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:31.390237 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d34600-01df-46e1-bd70-d2ebedb92fe1 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:31.391830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 162/322] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:59:31.396779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8de268-a495-44c7-9de6-dc3fea3c49ba None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:31.397069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8de268-a495-44c7-9de6-dc3fea3c49ba None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:31.397351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8de268-a495-44c7-9de6-dc3fea3c49ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:31.415182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-cd8de268-a495-44c7-9de6-dc3fea3c49ba None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:31.416375 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-cd8de268-a495-44c7-9de6-dc3fea3c49ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:31.423756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561123b4-42dc-4471-85c2-946f143026c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:31.424425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561123b4-42dc-4471-85c2-946f143026c7 None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:31.424711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561123b4-42dc-4471-85c2-946f143026c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:31.424994 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561123b4-42dc-4471-85c2-946f143026c7 None admin] PATH_INFO: `/v3/groups` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:31.425672 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561123b4-42dc-4471-85c2-946f143026c7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:31.427141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561123b4-42dc-4471-85c2-946f143026c7 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:31.443771 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 161/323] 10.222.0.44 () {60 vars in 1134 bytes} [Wed Jul 20 08:59:31 2022] POST /identity/v3/groups => generated 226 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:59:31.450211 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:31.500008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 163/324] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:31.524052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561123b4-42dc-4471-85c2-946f143026c7 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:31.633210 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:31.637013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3a894-ed43-4924-9196-0bc6927ef4d3 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:31.637305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3a894-ed43-4924-9196-0bc6927ef4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:31.637588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3a894-ed43-4924-9196-0bc6927ef4d3 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:31.639092 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 164/325] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:31.644367 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c4306-802e-464c-9995-783780cd29b4 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:31.644654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c4306-802e-464c-9995-783780cd29b4 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:31.644934 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c4306-802e-464c-9995-783780cd29b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:31.650889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561123b4-42dc-4471-85c2-946f143026c7 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:31.665276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-f29c4306-802e-464c-9995-783780cd29b4 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:31.667460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f29c4306-802e-464c-9995-783780cd29b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:31.689292 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c4aff-e084-4a2c-a888-3c087b33bbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:31.690523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c4aff-e084-4a2c-a888-3c087b33bbaa None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:31.691076 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c4aff-e084-4a2c-a888-3c087b33bbaa None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:31.691547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c4aff-e084-4a2c-a888-3c087b33bbaa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:31.707346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c4aff-e084-4a2c-a888-3c087b33bbaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:31.715265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-c52c4aff-e084-4a2c-a888-3c087b33bbaa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:31.715265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:31.715265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:31.715265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:31.715265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:31.715265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:31.715265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:31.715265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:31.715265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:31.715265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:31.715265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:31.715265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:31.715265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:31.715265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:31.715265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:31.715265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:31.716178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:31.716178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:31.716178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:31.716178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:31.716178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:31.716178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:31.716178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:31.716178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:31.716178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:31.716178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:31.716178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:31.716178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:31.716178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:31.716178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:31.716178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:31.716178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:31.716961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:31.716961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:31.716961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:31.716961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:31.716961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:31.716961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:31.716961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:31.716961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:31.716961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:31.716961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:31.716961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:31.716961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:31.716961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 162/326] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:31.726172 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c4aff-e084-4a2c-a888-3c087b33bbaa None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:31.751352 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 165/327] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:31.775636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c4306-802e-464c-9995-783780cd29b4 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:31.855436 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c4aff-e084-4a2c-a888-3c087b33bbaa None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:31.898658 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:31.902260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befaacfc-52d7-4082-878c-7dfab4a59387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:31.903225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befaacfc-52d7-4082-878c-7dfab4a59387 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:31.903519 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befaacfc-52d7-4082-878c-7dfab4a59387 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:31.903804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befaacfc-52d7-4082-878c-7dfab4a59387 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:31.905380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befaacfc-52d7-4082-878c-7dfab4a59387 None admin] RBAC: Adding query filter params (name=member) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:31.906231 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befaacfc-52d7-4082-878c-7dfab4a59387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:31.907804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befaacfc-52d7-4082-878c-7dfab4a59387 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:31.910236 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c4306-802e-464c-9995-783780cd29b4 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:31.917399 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 163/328] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:31.924514 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befaacfc-52d7-4082-878c-7dfab4a59387 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:31.936769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:31.953213 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27de84-26d0-45e1-b06c-bf7d4cd286b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:31.953930 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27de84-26d0-45e1-b06c-bf7d4cd286b6 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:31.954258 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27de84-26d0-45e1-b06c-bf7d4cd286b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:31.954509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27de84-26d0-45e1-b06c-bf7d4cd286b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:31.973587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27de84-26d0-45e1-b06c-bf7d4cd286b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:31.984270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-6f27de84-26d0-45e1-b06c-bf7d4cd286b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:31.984270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:31.984270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:31.984270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:31.984270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:31.984270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:31.984270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:31.984270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:31.984270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:31.984270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:31.984270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:31.984270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:31.984270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:31.984270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:31.984270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:31.984270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:31.985207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:31.985207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:31.985207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:31.985207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:31.985207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:31.985207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:31.985207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:31.985207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:31.985207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:31.985207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:31.985207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:31.985207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:31.985207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:31.985207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:31.985207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:31.985207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:31.986002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:31.986002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:31.986002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:31.986002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:31.986002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:31.986002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:31.986002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:31.986002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:31.986002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:31.986002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:31.986002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:31.986002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:31.986603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 166/329] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:31.994093 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27de84-26d0-45e1-b06c-bf7d4cd286b6 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:32.053822 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befaacfc-52d7-4082-878c-7dfab4a59387 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:32.100493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658fb62-8c72-41a4-bd83-281ecbd93c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:32.101769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658fb62-8c72-41a4-bd83-281ecbd93c2f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:32.102057 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658fb62-8c72-41a4-bd83-281ecbd93c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:32.102341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658fb62-8c72-41a4-bd83-281ecbd93c2f None admin] PATH_INFO: `/v3/users/e5584d0226e943469166fe4d3764b3d5` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:32.103614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4658fb62-8c72-41a4-bd83-281ecbd93c2f None admin] RBAC: Authorizing `identity:get_user(user_id=e5584d0226e943469166fe4d3764b3d5)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:32.105910 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4658fb62-8c72-41a4-bd83-281ecbd93c2f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:32.109181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 164/330] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:31 2022] GET /identity/v3/users/e5584d0226e943469166fe4d3764b3d5 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:32.115095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658fb62-8c72-41a4-bd83-281ecbd93c2f None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:32.125241 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27de84-26d0-45e1-b06c-bf7d4cd286b6 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:32.165784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71041e86-6b7a-42b6-82f2-edeaa2228792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:32.166454 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71041e86-6b7a-42b6-82f2-edeaa2228792 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:32.166762 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71041e86-6b7a-42b6-82f2-edeaa2228792 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:32.167041 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71041e86-6b7a-42b6-82f2-edeaa2228792 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:32.167840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71041e86-6b7a-42b6-82f2-edeaa2228792 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:32.168181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71041e86-6b7a-42b6-82f2-edeaa2228792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:32.170369 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71041e86-6b7a-42b6-82f2-edeaa2228792 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:32.183084 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 167/331] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:32.191316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71041e86-6b7a-42b6-82f2-edeaa2228792 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:32.210852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:32.217114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:32.245617 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658fb62-8c72-41a4-bd83-281ecbd93c2f None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:32.284267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ea428-e5e6-43fe-8cee-2e87ea567ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:32.284935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ea428-e5e6-43fe-8cee-2e87ea567ccb None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:32.285329 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ea428-e5e6-43fe-8cee-2e87ea567ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:32.285618 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ea428-e5e6-43fe-8cee-2e87ea567ccb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:32.286470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ea428-e5e6-43fe-8cee-2e87ea567ccb None admin] RBAC: Adding query filter params (role.id=0e2011aac3e1430cb96bdbf810e89eb7, user.id=e5584d0226e943469166fe4d3764b3d5, scope.system=all) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:32.286919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ea428-e5e6-43fe-8cee-2e87ea567ccb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:32.288373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ea428-e5e6-43fe-8cee-2e87ea567ccb None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:32.304066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 165/332] 10.222.0.44 () {56 vars in 1308 bytes} [Wed Jul 20 08:59:32 2022] GET /identity/v3/role_assignments?user.id=e5584d0226e943469166fe4d3764b3d5&scope.system=all&role.id=0e2011aac3e1430cb96bdbf810e89eb7 => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:32.313942 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71041e86-6b7a-42b6-82f2-edeaa2228792 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:32.356066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aaf416-e36a-482e-ace3-a2843a52846c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:32.357011 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaf416-e36a-482e-ace3-a2843a52846c None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:32.357348 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaf416-e36a-482e-ace3-a2843a52846c None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:32.357635 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaf416-e36a-482e-ace3-a2843a52846c None admin] PATH_INFO: `/v3/users/c62cb7d9924c41dbac763b2f862c6285` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:32.359261 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aaf416-e36a-482e-ace3-a2843a52846c None admin] RBAC: Authorizing `identity:get_user(user_id=c62cb7d9924c41dbac763b2f862c6285)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:32.360735 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aaf416-e36a-482e-ace3-a2843a52846c None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:32.366855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 168/333] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:32 2022] GET /identity/v3/users/c62cb7d9924c41dbac763b2f862c6285 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:32.368663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ea428-e5e6-43fe-8cee-2e87ea567ccb None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:32.490415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ea428-e5e6-43fe-8cee-2e87ea567ccb None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:32.533023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aca89a-7d06-4c56-8f5e-2832d746f028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:32.533900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aca89a-7d06-4c56-8f5e-2832d746f028 None admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:32.534185 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aca89a-7d06-4c56-8f5e-2832d746f028 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:32.534473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aca89a-7d06-4c56-8f5e-2832d746f028 None admin] PATH_INFO: `/v3/system/users/c62cb7d9924c41dbac763b2f862c6285/roles/559659e5dbc346d79735375b9130736d` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:32.540671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aca89a-7d06-4c56-8f5e-2832d746f028 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c62cb7d9924c41dbac763b2f862c6285, role_id=559659e5dbc346d79735375b9130736d)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:32.542075 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aca89a-7d06-4c56-8f5e-2832d746f028 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:32.543829 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:32.563946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 166/334] 10.222.0.44 () {58 vars in 1334 bytes} [Wed Jul 20 08:59:32 2022] PUT /identity/v3/system/users/c62cb7d9924c41dbac763b2f862c6285/roles/559659e5dbc346d79735375b9130736d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:59:33.920174 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f3843-e270-4427-9fd8-83b43d35f60b None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:33.920752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f3843-e270-4427-9fd8-83b43d35f60b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:33.920885 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f3843-e270-4427-9fd8-83b43d35f60b None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:33.922340 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 169/335] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:33.927893 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ec32b-9de8-4052-8fad-97a3aea94f01 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:33.928311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ec32b-9de8-4052-8fad-97a3aea94f01 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:33.928483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ec32b-9de8-4052-8fad-97a3aea94f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:33.951743 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-d80ec32b-9de8-4052-8fad-97a3aea94f01 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:33.953548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d80ec32b-9de8-4052-8fad-97a3aea94f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:34.039299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 167/336] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:34.080189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f3843-e270-4427-9fd8-83b43d35f60b None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:34.181206 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:34.205970 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f3843-e270-4427-9fd8-83b43d35f60b None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:34.244445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631d147-288f-4c93-9b43-6f80967229c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:34.245254 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631d147-288f-4c93-9b43-6f80967229c7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:34.245531 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631d147-288f-4c93-9b43-6f80967229c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:34.245800 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631d147-288f-4c93-9b43-6f80967229c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:34.258984 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631d147-288f-4c93-9b43-6f80967229c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:34.271605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-e631d147-288f-4c93-9b43-6f80967229c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:34.271605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:34.271605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:34.271605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:34.271605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:34.271605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:34.271605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:34.271605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:34.271605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:34.271605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:34.271605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:34.271605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:34.271605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:34.271605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:34.271605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:34.271605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:34.272509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:34.272509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:34.272509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:34.272509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:34.272509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:34.272509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:34.272509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:34.272509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:34.272509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:34.272509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:34.272509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:34.272509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:34.272509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:34.272509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:34.272509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:34.272509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:34.273306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:34.273306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:34.273306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:34.273306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:34.273306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:34.273306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:34.273306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:34.273306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:34.273306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:34.273306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:34.273306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:34.273306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:34.273306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 170/337] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:34.279026 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ec32b-9de8-4052-8fad-97a3aea94f01 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:34.428134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ec32b-9de8-4052-8fad-97a3aea94f01 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:34.461473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:34.471887 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854635c-f2ca-4632-abc4-ea14d7e37e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:34.472632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854635c-f2ca-4632-abc4-ea14d7e37e72 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:34.472918 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854635c-f2ca-4632-abc4-ea14d7e37e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:34.473266 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854635c-f2ca-4632-abc4-ea14d7e37e72 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:34.474044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854635c-f2ca-4632-abc4-ea14d7e37e72 None admin] RBAC: Adding query filter params (name=member) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:34.474354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854635c-f2ca-4632-abc4-ea14d7e37e72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:34.476140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854635c-f2ca-4632-abc4-ea14d7e37e72 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:34.489101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 168/338] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:34.494510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631d147-288f-4c93-9b43-6f80967229c7 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:34.633158 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f56a6-1605-42a9-9c4a-750eef449465 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:34.633476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f56a6-1605-42a9-9c4a-750eef449465 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:34.633727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f56a6-1605-42a9-9c4a-750eef449465 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:34.635414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 169/339] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:59:34.638880 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631d147-288f-4c93-9b43-6f80967229c7 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:34.641384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed8c3b-2cb7-49f8-9520-34da84c33414 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:34.641796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed8c3b-2cb7-49f8-9520-34da84c33414 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:34.642084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed8c3b-2cb7-49f8-9520-34da84c33414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:34.680050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-abed8c3b-2cb7-49f8-9520-34da84c33414 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:34.680552 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f44de-ee9e-4677-897b-66efe51278dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:34.681220 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f44de-ee9e-4677-897b-66efe51278dc None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:34.681506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f44de-ee9e-4677-897b-66efe51278dc None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:34.681788 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f44de-ee9e-4677-897b-66efe51278dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:34.682320 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-abed8c3b-2cb7-49f8-9520-34da84c33414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:34.715128 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:34.779740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 170/340] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:34.783813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa7ad7-c3e6-490f-8701-9e05992ddc5e None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:34.784728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f44de-ee9e-4677-897b-66efe51278dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:34.785191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa7ad7-c3e6-490f-8701-9e05992ddc5e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:34.785666 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa7ad7-c3e6-490f-8701-9e05992ddc5e None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:34.787335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f44de-ee9e-4677-897b-66efe51278dc None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:34.788889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 171/341] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:34.795312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6fa0f-a4c4-4941-b50a-2ffccfd779b0 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:34.796381 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6fa0f-a4c4-4941-b50a-2ffccfd779b0 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:34.796833 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6fa0f-a4c4-4941-b50a-2ffccfd779b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:34.825747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-a3a6fa0f-a4c4-4941-b50a-2ffccfd779b0 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:34.828128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-a3a6fa0f-a4c4-4941-b50a-2ffccfd779b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:34.972317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 172/342] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:34.973130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6fa0f-a4c4-4941-b50a-2ffccfd779b0 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:34.982794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 171/343] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:34.990338 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f44de-ee9e-4677-897b-66efe51278dc None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:35.051370 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:35.091111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:35.142671 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f44de-ee9e-4677-897b-66efe51278dc None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:35.181091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40556612-f8d6-40c7-8d01-271518802fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:35.181951 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40556612-f8d6-40c7-8d01-271518802fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:35.182399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40556612-f8d6-40c7-8d01-271518802fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:35.182691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40556612-f8d6-40c7-8d01-271518802fa4 None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:35.183875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40556612-f8d6-40c7-8d01-271518802fa4 None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:35.185305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40556612-f8d6-40c7-8d01-271518802fa4 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:35.186565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6fa0f-a4c4-4941-b50a-2ffccfd779b0 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:35.188796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 172/344] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:34 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:35.189596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40556612-f8d6-40c7-8d01-271518802fa4 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:35.260091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b604d-6e0b-432f-a6e9-c6824bb2de40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:35.260937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b604d-6e0b-432f-a6e9-c6824bb2de40 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:35.261295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b604d-6e0b-432f-a6e9-c6824bb2de40 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:35.261585 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b604d-6e0b-432f-a6e9-c6824bb2de40 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:35.298392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b604d-6e0b-432f-a6e9-c6824bb2de40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:35.312574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-490b604d-6e0b-432f-a6e9-c6824bb2de40 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:35.312574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:35.312574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:35.312574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:35.312574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:35.312574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:35.312574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:35.312574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:35.312574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:35.312574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:35.312574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:35.312574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:35.312574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:35.312574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:35.312574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:35.312574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:35.313499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:35.313499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:35.313499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:35.313499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:35.313499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:35.313499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:35.313499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:35.313499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:35.313499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:35.313499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:35.313499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:35.313499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:35.313499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:35.313499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:35.313499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:35.313499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:35.315060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:35.315060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:35.315060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:35.315060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:35.315060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:35.315060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:35.315060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:35.315060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:35.315060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:35.315060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:35.315060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:35.315060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:35.315060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 173/345] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 08:59:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:35.315060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b604d-6e0b-432f-a6e9-c6824bb2de40 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:35.322472 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40556612-f8d6-40c7-8d01-271518802fa4 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:35.339175 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:35.364502 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8c711-e040-4fb3-80e0-2cf17f6cb8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:35.365417 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8c711-e040-4fb3-80e0-2cf17f6cb8fa None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:35.365770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8c711-e040-4fb3-80e0-2cf17f6cb8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:35.366113 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e8c711-e040-4fb3-80e0-2cf17f6cb8fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:35.377713 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e8c711-e040-4fb3-80e0-2cf17f6cb8fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:35.385899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-61e8c711-e040-4fb3-80e0-2cf17f6cb8fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:35.385899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:35.385899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:35.385899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:35.385899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:35.385899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:35.385899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:35.385899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:35.385899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:35.385899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:35.385899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:35.385899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:35.385899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:35.385899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:35.385899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:35.385899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:35.386828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:35.386828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:35.386828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:35.386828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:35.386828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:35.386828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:35.386828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:35.386828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:35.386828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:35.386828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:35.386828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:35.386828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:35.386828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:35.386828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:35.386828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:35.386828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:35.387728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:35.387728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:35.387728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:35.387728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:35.387728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:35.387728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:35.387728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:35.387728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:35.387728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:35.387728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:35.387728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:35.387728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:35.387728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 173/346] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:35.388384 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8c711-e040-4fb3-80e0-2cf17f6cb8fa None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:35.493728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e8c711-e040-4fb3-80e0-2cf17f6cb8fa None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:35.510513 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b604d-6e0b-432f-a6e9-c6824bb2de40 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:35.528210 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:35.528503 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81322b-25e6-4a0e-96bd-020db8568f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:35.529183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81322b-25e6-4a0e-96bd-020db8568f6e None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:35.529483 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81322b-25e6-4a0e-96bd-020db8568f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:35.529737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81322b-25e6-4a0e-96bd-020db8568f6e None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:35.532319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81322b-25e6-4a0e-96bd-020db8568f6e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:35.532319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81322b-25e6-4a0e-96bd-020db8568f6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:35.533172 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81322b-25e6-4a0e-96bd-020db8568f6e None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:35.542023 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 174/347] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:35.544192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81322b-25e6-4a0e-96bd-020db8568f6e None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:35.578569 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a1103-3b57-4e75-bfb0-27802d489dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:35.578917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:35.580446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a1103-3b57-4e75-bfb0-27802d489dde None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:35.580736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a1103-3b57-4e75-bfb0-27802d489dde None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:35.581011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a1103-3b57-4e75-bfb0-27802d489dde None admin] PATH_INFO: `/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:35.592189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a1103-3b57-4e75-bfb0-27802d489dde None admin] RBAC: Authorizing `identity:get_group(group_id=a7c4a8842bf64b77a6ed9d6b706b3c62)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:35.594213 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a1103-3b57-4e75-bfb0-27802d489dde None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:35.596021 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 174/348] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jul 20 08:59:35 2022] GET /identity/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:35.596803 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a1103-3b57-4e75-bfb0-27802d489dde None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:35.654897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81322b-25e6-4a0e-96bd-020db8568f6e None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:35.690944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32527f3-cd30-4898-b4a5-d1ad20a09221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:35.691675 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32527f3-cd30-4898-b4a5-d1ad20a09221 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:35.692028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32527f3-cd30-4898-b4a5-d1ad20a09221 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:35.692308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32527f3-cd30-4898-b4a5-d1ad20a09221 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:35.693060 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32527f3-cd30-4898-b4a5-d1ad20a09221 None admin] RBAC: Adding query filter params (name=member) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:35.693364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32527f3-cd30-4898-b4a5-d1ad20a09221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:35.695144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32527f3-cd30-4898-b4a5-d1ad20a09221 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:35.706659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 175/349] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:35.707516 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32527f3-cd30-4898-b4a5-d1ad20a09221 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:35.755409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a1103-3b57-4e75-bfb0-27802d489dde None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:35.801891 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159284c-f057-44bc-a960-b1f6741a618c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:35.802780 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159284c-f057-44bc-a960-b1f6741a618c None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:35.803072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159284c-f057-44bc-a960-b1f6741a618c None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:35.803356 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159284c-f057-44bc-a960-b1f6741a618c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:35.844066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:35.851512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32527f3-cd30-4898-b4a5-d1ad20a09221 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:35.857435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:35.884994 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159284c-f057-44bc-a960-b1f6741a618c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:35.886517 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159284c-f057-44bc-a960-b1f6741a618c None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:35.891812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9e62c-183a-4c47-a2cf-8bb3e18f8b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:35.892542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9e62c-183a-4c47-a2cf-8bb3e18f8b95 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:35.892831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9e62c-183a-4c47-a2cf-8bb3e18f8b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:35.893166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9e62c-183a-4c47-a2cf-8bb3e18f8b95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:35.894014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9e62c-183a-4c47-a2cf-8bb3e18f8b95 None admin] RBAC: Adding query filter params (group.id=a7c4a8842bf64b77a6ed9d6b706b3c62, role.id=0e2011aac3e1430cb96bdbf810e89eb7, scope.project.id=81fa154d1fc04266abee9f0051cba2fa) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:35.894331 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9e62c-183a-4c47-a2cf-8bb3e18f8b95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:35.895816 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9e62c-183a-4c47-a2cf-8bb3e18f8b95 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:35.902153 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 176/350] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jul 20 08:59:35 2022] GET /identity/v3/role_assignments?group.id=a7c4a8842bf64b77a6ed9d6b706b3c62&scope.project.id=81fa154d1fc04266abee9f0051cba2fa&role.id=0e2011aac3e1430cb96bdbf810e89eb7 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:35.907203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9e62c-183a-4c47-a2cf-8bb3e18f8b95 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:36.001863 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 175/351] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:36.003259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77006360-ec27-402b-a246-970e685bdfd4 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:36.003547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77006360-ec27-402b-a246-970e685bdfd4 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:36.003832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77006360-ec27-402b-a246-970e685bdfd4 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:36.005378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 176/352] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:36.007299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77006360-ec27-402b-a246-970e685bdfd4 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:36.078787 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9e62c-183a-4c47-a2cf-8bb3e18f8b95 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:36.098736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:36.128586 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c73483-8a87-4a7c-b58a-93ec974d47ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:36.129526 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c73483-8a87-4a7c-b58a-93ec974d47ca None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:36.129810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c73483-8a87-4a7c-b58a-93ec974d47ca None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:36.130090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c73483-8a87-4a7c-b58a-93ec974d47ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:36.132141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77006360-ec27-402b-a246-970e685bdfd4 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:36.169728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:36.186759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbe5fb-0dcb-4632-a966-c724119dafe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:36.187991 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbe5fb-0dcb-4632-a966-c724119dafe0 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:36.188457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbe5fb-0dcb-4632-a966-c724119dafe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:36.189037 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbe5fb-0dcb-4632-a966-c724119dafe0 None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:36.190333 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbe5fb-0dcb-4632-a966-c724119dafe0 None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:36.191980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbe5fb-0dcb-4632-a966-c724119dafe0 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:36.194836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 177/353] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:36 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:36.201278 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d8be4-9214-42cb-953a-56f8916f14ca None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:36.202011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d8be4-9214-42cb-953a-56f8916f14ca None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:36.202815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d8be4-9214-42cb-953a-56f8916f14ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:36.220604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-f30d8be4-9214-42cb-953a-56f8916f14ca None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:36.220788 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c73483-8a87-4a7c-b58a-93ec974d47ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:36.222178 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c73483-8a87-4a7c-b58a-93ec974d47ca None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:36.222395 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f30d8be4-9214-42cb-953a-56f8916f14ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:36.297180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 178/354] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:36.298891 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b374344-6d68-42f7-8cac-f7cf5c659240 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:36.299382 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b374344-6d68-42f7-8cac-f7cf5c659240 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:36.299856 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b374344-6d68-42f7-8cac-f7cf5c659240 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:36.301410 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 179/355] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:36.302390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b374344-6d68-42f7-8cac-f7cf5c659240 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:36.355512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 177/356] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:36.356820 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc6038-9352-4cd7-8feb-8695439ea953 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:36.357113 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc6038-9352-4cd7-8feb-8695439ea953 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:36.357399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc6038-9352-4cd7-8feb-8695439ea953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:36.387629 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a9bc6038-9352-4cd7-8feb-8695439ea953 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:36.388800 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a9bc6038-9352-4cd7-8feb-8695439ea953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:36.389069 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:36.422174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b374344-6d68-42f7-8cac-f7cf5c659240 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:36.455686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:36.460730 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52959e-dead-47e6-8089-a0ca46853213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:36.461915 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52959e-dead-47e6-8089-a0ca46853213 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:36.462390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52959e-dead-47e6-8089-a0ca46853213 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:36.462881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52959e-dead-47e6-8089-a0ca46853213 None admin] PATH_INFO: `/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:36.464281 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52959e-dead-47e6-8089-a0ca46853213 None admin] RBAC: Authorizing `identity:get_group(group_id=a7c4a8842bf64b77a6ed9d6b706b3c62)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:36.465969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52959e-dead-47e6-8089-a0ca46853213 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:36.468236 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 180/357] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jul 20 08:59:36 2022] GET /identity/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:36.469483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52959e-dead-47e6-8089-a0ca46853213 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:36.476557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 178/358] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:36.477320 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc6038-9352-4cd7-8feb-8695439ea953 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:36.597687 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52959e-dead-47e6-8089-a0ca46853213 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:36.608658 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc6038-9352-4cd7-8feb-8695439ea953 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:36.637667 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422da796-d298-455d-9778-b15b941e08b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:36.638781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422da796-d298-455d-9778-b15b941e08b4 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:36.639068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422da796-d298-455d-9778-b15b941e08b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:36.640214 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422da796-d298-455d-9778-b15b941e08b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:36.652269 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1181bdb-88a7-4fe7-8169-1b7aa0833389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:36.653132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1181bdb-88a7-4fe7-8169-1b7aa0833389 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:36.653380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:36.653492 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1181bdb-88a7-4fe7-8169-1b7aa0833389 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:36.653683 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1181bdb-88a7-4fe7-8169-1b7aa0833389 None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:36.654661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422da796-d298-455d-9778-b15b941e08b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:36.655545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1181bdb-88a7-4fe7-8169-1b7aa0833389 None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:36.657340 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1181bdb-88a7-4fe7-8169-1b7aa0833389 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:36.659778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 179/359] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:36 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:36.660163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1181bdb-88a7-4fe7-8169-1b7aa0833389 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:36.666184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-422da796-d298-455d-9778-b15b941e08b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:36.666184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:36.666184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:36.666184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:36.666184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:36.666184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:36.666184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:36.666184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:36.666184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:36.666184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:36.666184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:36.666184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:36.666184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:36.666184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:36.666184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:36.666184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:36.667166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:36.667166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:36.667166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:36.667166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:36.667166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:36.667166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:36.667166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:36.667166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:36.667166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:36.667166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:36.667166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:36.667166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:36.667166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:36.667166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:36.667166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:36.667166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:36.667973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:36.667973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:36.667973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:36.667973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:36.667973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:36.667973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:36.667973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:36.667973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:36.667973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:36.667973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:36.667973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:36.667973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:36.667973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 181/360] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 08:59:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:36.669305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bf23c-4653-4609-8516-b82269385afd None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:36.669593 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bf23c-4653-4609-8516-b82269385afd None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:36.669871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bf23c-4653-4609-8516-b82269385afd None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:36.676005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 182/361] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:36.676005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bf23c-4653-4609-8516-b82269385afd None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:36.737063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:36.792501 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1181bdb-88a7-4fe7-8169-1b7aa0833389 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:36.807801 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bf23c-4653-4609-8516-b82269385afd None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:36.837399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d40515-8033-4322-8d2d-704818862a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:36.838177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d40515-8033-4322-8d2d-704818862a77 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:36.838530 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d40515-8033-4322-8d2d-704818862a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:36.838917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d40515-8033-4322-8d2d-704818862a77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:36.839827 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d40515-8033-4322-8d2d-704818862a77 None admin] RBAC: Adding query filter params (group.id=a7c4a8842bf64b77a6ed9d6b706b3c62, role.id=05ca4843c34e463c94de09f756d95318, scope.project.id=81fa154d1fc04266abee9f0051cba2fa) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:36.840144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d40515-8033-4322-8d2d-704818862a77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:36.841580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d40515-8033-4322-8d2d-704818862a77 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:36.847921 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 180/362] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jul 20 08:59:36 2022] GET /identity/v3/role_assignments?group.id=a7c4a8842bf64b77a6ed9d6b706b3c62&scope.project.id=81fa154d1fc04266abee9f0051cba2fa&role.id=05ca4843c34e463c94de09f756d95318 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:36.848740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d40515-8033-4322-8d2d-704818862a77 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:36.858071 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b1aa7-6458-4054-b083-e6dfe1c0a73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:36.860117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b1aa7-6458-4054-b083-e6dfe1c0a73d None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:36.860648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b1aa7-6458-4054-b083-e6dfe1c0a73d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:36.861138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b1aa7-6458-4054-b083-e6dfe1c0a73d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:36.878550 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b1aa7-6458-4054-b083-e6dfe1c0a73d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:36.888972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-342b1aa7-6458-4054-b083-e6dfe1c0a73d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:36.888972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:36.888972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:36.888972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:36.888972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:36.888972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:36.888972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:36.888972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:36.888972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:36.888972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:36.888972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:36.888972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:36.888972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:36.888972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:36.888972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:36.888972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:36.890522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:36.890522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:36.890522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:36.890522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:36.890522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:36.890522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:36.890522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:36.890522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:36.890522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:36.890522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:36.890522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:36.890522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:36.890522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:36.890522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:36.890522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:36.890522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:36.891364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:36.891364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:36.891364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:36.891364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:36.891364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:36.891364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:36.891364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:36.891364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:36.891364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:36.891364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:36.891364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:59:36.891364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:36.893781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 183/363] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:36.895828 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3861bbb-658d-4724-aeff-24297390e645 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:36.895828 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3861bbb-658d-4724-aeff-24297390e645 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:36.896127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3861bbb-658d-4724-aeff-24297390e645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:36.921694 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-d3861bbb-658d-4724-aeff-24297390e645 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:36.923878 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d3861bbb-658d-4724-aeff-24297390e645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:36.981410 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d40515-8033-4322-8d2d-704818862a77 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:36.999818 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:37.013746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 184/364] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:37.016264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3861bbb-658d-4724-aeff-24297390e645 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:37.019337 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:37.020069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5960c-5b57-4049-97f3-0a829561c40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:37.021231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5960c-5b57-4049-97f3-0a829561c40a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:37.021533 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5960c-5b57-4049-97f3-0a829561c40a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:37.022000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5960c-5b57-4049-97f3-0a829561c40a None admin] PATH_INFO: `/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:37.023399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5960c-5b57-4049-97f3-0a829561c40a None admin] RBAC: Authorizing `identity:get_group(group_id=a7c4a8842bf64b77a6ed9d6b706b3c62)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:37.024956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5960c-5b57-4049-97f3-0a829561c40a None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:37.027845 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 181/365] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jul 20 08:59:36 2022] GET /identity/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:37.028799 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5960c-5b57-4049-97f3-0a829561c40a None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:37.138755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5960c-5b57-4049-97f3-0a829561c40a None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:37.153582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3861bbb-658d-4724-aeff-24297390e645 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:37.174931 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204ebbd-5d84-4a10-9fdd-a76bb02031e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:37.175602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204ebbd-5d84-4a10-9fdd-a76bb02031e9 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:37.175885 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204ebbd-5d84-4a10-9fdd-a76bb02031e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:37.176162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204ebbd-5d84-4a10-9fdd-a76bb02031e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:37.176903 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204ebbd-5d84-4a10-9fdd-a76bb02031e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:37.177205 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204ebbd-5d84-4a10-9fdd-a76bb02031e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:37.178568 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204ebbd-5d84-4a10-9fdd-a76bb02031e9 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:37.187591 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 182/366] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:37.188425 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204ebbd-5d84-4a10-9fdd-a76bb02031e9 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:37.198273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58873ee8-2792-4ea9-a195-565d32e07eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:37.199002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58873ee8-2792-4ea9-a195-565d32e07eab None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:37.199421 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58873ee8-2792-4ea9-a195-565d32e07eab None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:37.199707 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58873ee8-2792-4ea9-a195-565d32e07eab None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:37.200468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58873ee8-2792-4ea9-a195-565d32e07eab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:37.200776 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58873ee8-2792-4ea9-a195-565d32e07eab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:37.202969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58873ee8-2792-4ea9-a195-565d32e07eab None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:37.212119 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 185/367] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:37.212773 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58873ee8-2792-4ea9-a195-565d32e07eab None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:37.244652 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:37.274995 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:37.299321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204ebbd-5d84-4a10-9fdd-a76bb02031e9 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:37.327538 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58873ee8-2792-4ea9-a195-565d32e07eab None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:37.339603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae4b9a-90a7-4bfc-9d11-981b5a2f3d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:37.340281 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae4b9a-90a7-4bfc-9d11-981b5a2f3d25 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:37.340577 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae4b9a-90a7-4bfc-9d11-981b5a2f3d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:37.340834 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae4b9a-90a7-4bfc-9d11-981b5a2f3d25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:37.341682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae4b9a-90a7-4bfc-9d11-981b5a2f3d25 None admin] RBAC: Adding query filter params (group.id=a7c4a8842bf64b77a6ed9d6b706b3c62, role.id=0e2011aac3e1430cb96bdbf810e89eb7, scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:37.341995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae4b9a-90a7-4bfc-9d11-981b5a2f3d25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:37.343754 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae4b9a-90a7-4bfc-9d11-981b5a2f3d25 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:37.350591 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 183/368] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jul 20 08:59:37 2022] GET /identity/v3/role_assignments?group.id=a7c4a8842bf64b77a6ed9d6b706b3c62&scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7&role.id=0e2011aac3e1430cb96bdbf810e89eb7 => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:37.351916 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae4b9a-90a7-4bfc-9d11-981b5a2f3d25 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:37.369179 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f73c8-7841-42a1-b5d8-73e3c38b0a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:37.370366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f73c8-7841-42a1-b5d8-73e3c38b0a7f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:37.370868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f73c8-7841-42a1-b5d8-73e3c38b0a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:37.371288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f73c8-7841-42a1-b5d8-73e3c38b0a7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:37.385810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f73c8-7841-42a1-b5d8-73e3c38b0a7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:37.399403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-ce5f73c8-7841-42a1-b5d8-73e3c38b0a7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:37.399403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:37.399403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:37.399403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:37.399403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:37.399403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:37.399403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:37.399403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:37.399403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:37.399403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:37.399403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:37.399403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:37.399403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:37.399403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:37.399403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:37.399403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:37.400366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:37.400366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:37.400366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:37.400366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:37.400366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:37.400366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:37.400366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:37.400366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:37.400366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:37.400366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:37.400366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:37.400366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:37.400366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:37.400366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:37.400366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:37.400366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:37.401174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:37.401174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:37.401174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:37.401174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:37.401174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:37.401174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:37.401174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:37.401174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:37.401174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:37.401174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:37.401174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:37.401174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:37.401174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 186/369] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 08:59:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:37.407350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f73c8-7841-42a1-b5d8-73e3c38b0a7f None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:37.524443 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae4b9a-90a7-4bfc-9d11-981b5a2f3d25 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:37.553688 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:37.590865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b5dea-2ef7-4ccd-bb7b-658e5e85f619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:37.591756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b5dea-2ef7-4ccd-bb7b-658e5e85f619 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:37.592076 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b5dea-2ef7-4ccd-bb7b-658e5e85f619 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:37.593016 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b5dea-2ef7-4ccd-bb7b-658e5e85f619 None admin] PATH_INFO: `/v3/users/c62cb7d9924c41dbac763b2f862c6285` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:37.595660 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b5dea-2ef7-4ccd-bb7b-658e5e85f619 None admin] RBAC: Authorizing `identity:get_user(user_id=c62cb7d9924c41dbac763b2f862c6285)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:37.597566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b5dea-2ef7-4ccd-bb7b-658e5e85f619 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:37.599766 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 184/370] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 08:59:37 2022] GET /identity/v3/users/c62cb7d9924c41dbac763b2f862c6285 => generated 293 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:37.600911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b5dea-2ef7-4ccd-bb7b-658e5e85f619 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:37.611816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f73c8-7841-42a1-b5d8-73e3c38b0a7f None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:37.633887 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:37.675524 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee0ebe-bbd6-4eec-b7e3-5cde2af9d142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:37.676124 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee0ebe-bbd6-4eec-b7e3-5cde2af9d142 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:37.676420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee0ebe-bbd6-4eec-b7e3-5cde2af9d142 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:37.676709 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee0ebe-bbd6-4eec-b7e3-5cde2af9d142 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:37.774047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b5dea-2ef7-4ccd-bb7b-658e5e85f619 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:37.814187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2117b21-1bd0-4b9a-951a-9277e28d7767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:37.814904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2117b21-1bd0-4b9a-951a-9277e28d7767 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:37.815191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2117b21-1bd0-4b9a-951a-9277e28d7767 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:37.815472 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2117b21-1bd0-4b9a-951a-9277e28d7767 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:37.816232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2117b21-1bd0-4b9a-951a-9277e28d7767 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:37.816540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2117b21-1bd0-4b9a-951a-9277e28d7767 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:37.819674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2117b21-1bd0-4b9a-951a-9277e28d7767 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:37.828620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee0ebe-bbd6-4eec-b7e3-5cde2af9d142 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:37.830380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee0ebe-bbd6-4eec-b7e3-5cde2af9d142 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:37.831240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 185/371] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:37.832100 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2117b21-1bd0-4b9a-951a-9277e28d7767 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:37.892892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:37.989985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2117b21-1bd0-4b9a-951a-9277e28d7767 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:38.028080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 187/372] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:38.028962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee0ebe-bbd6-4eec-b7e3-5cde2af9d142 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:38.053123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:38.057358 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dca41-8b6c-4edf-94e6-6754c6e0e7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:38.058081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dca41-8b6c-4edf-94e6-6754c6e0e7bb None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:38.058411 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dca41-8b6c-4edf-94e6-6754c6e0e7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:38.058703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dca41-8b6c-4edf-94e6-6754c6e0e7bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:38.059592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dca41-8b6c-4edf-94e6-6754c6e0e7bb None admin] RBAC: Adding query filter params (role.id=559659e5dbc346d79735375b9130736d, user.id=c62cb7d9924c41dbac763b2f862c6285, scope.system=all) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:38.059904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dca41-8b6c-4edf-94e6-6754c6e0e7bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:38.061473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dca41-8b6c-4edf-94e6-6754c6e0e7bb None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:38.086061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 186/373] 10.222.0.44 () {56 vars in 1308 bytes} [Wed Jul 20 08:59:37 2022] GET /identity/v3/role_assignments?user.id=c62cb7d9924c41dbac763b2f862c6285&scope.system=all&role.id=559659e5dbc346d79735375b9130736d => generated 510 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:38.086923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dca41-8b6c-4edf-94e6-6754c6e0e7bb None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:38.240373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee0ebe-bbd6-4eec-b7e3-5cde2af9d142 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:38.327835 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dca41-8b6c-4edf-94e6-6754c6e0e7bb None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:38.329523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:38.340511 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bbf07-5ba8-41a3-bb0c-50688a8fbd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:38.341187 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bbf07-5ba8-41a3-bb0c-50688a8fbd89 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:38.341471 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bbf07-5ba8-41a3-bb0c-50688a8fbd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:38.341753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bbf07-5ba8-41a3-bb0c-50688a8fbd89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:38.406932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9f68b-c8f5-44bc-b067-056c75aaed6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:38.407808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9f68b-c8f5-44bc-b067-056c75aaed6b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:38.408093 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9f68b-c8f5-44bc-b067-056c75aaed6b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:38.408372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9f68b-c8f5-44bc-b067-056c75aaed6b None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:38.410518 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c9f68b-c8f5-44bc-b067-056c75aaed6b None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:38.412003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c9f68b-c8f5-44bc-b067-056c75aaed6b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:38.413722 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 187/374] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:38 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:38.420655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9f68b-c8f5-44bc-b067-056c75aaed6b None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:38.497203 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bbf07-5ba8-41a3-bb0c-50688a8fbd89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:38.498629 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bbf07-5ba8-41a3-bb0c-50688a8fbd89 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:38.518135 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:38.614401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9f68b-c8f5-44bc-b067-056c75aaed6b None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:38.675933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 188/375] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:38.677278 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc918-4f49-407f-beb4-696abcf248b1 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:38.677564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc918-4f49-407f-beb4-696abcf248b1 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:38.677676 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ed1ba-02c8-471a-a60d-2df992e027b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:38.677867 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebc918-4f49-407f-beb4-696abcf248b1 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:38.678479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ed1ba-02c8-471a-a60d-2df992e027b5 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:38.678790 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ed1ba-02c8-471a-a60d-2df992e027b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:38.679066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ed1ba-02c8-471a-a60d-2df992e027b5 None admin] PATH_INFO: `/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:38.680487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ed1ba-02c8-471a-a60d-2df992e027b5 None admin] RBAC: Authorizing `identity:get_group(group_id=a7c4a8842bf64b77a6ed9d6b706b3c62)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:38.682070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ed1ba-02c8-471a-a60d-2df992e027b5 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:38.683289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 189/376] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:38.687845 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebc918-4f49-407f-beb4-696abcf248b1 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:38.690426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 188/377] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jul 20 08:59:38 2022] GET /identity/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:38.692137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939a394-f30d-4f01-ae33-75f361bb4e98 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:38.692587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939a394-f30d-4f01-ae33-75f361bb4e98 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:38.693017 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939a394-f30d-4f01-ae33-75f361bb4e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:38.714150 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-e939a394-f30d-4f01-ae33-75f361bb4e98 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:38.716015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e939a394-f30d-4f01-ae33-75f361bb4e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:38.755662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:38.814716 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 189/378] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:38.814716 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939a394-f30d-4f01-ae33-75f361bb4e98 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:38.839461 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebc918-4f49-407f-beb4-696abcf248b1 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:38.859799 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:38.882346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb461303-0a60-438e-97e5-134eb283cf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:38.883605 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb461303-0a60-438e-97e5-134eb283cf38 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:38.884111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb461303-0a60-438e-97e5-134eb283cf38 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:38.884600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb461303-0a60-438e-97e5-134eb283cf38 None admin] PATH_INFO: `/v3/projects/d5e32bdaa14a4d31baa44b436b02a242` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:38.886014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb461303-0a60-438e-97e5-134eb283cf38 None admin] RBAC: Authorizing `identity:get_project(project_id=d5e32bdaa14a4d31baa44b436b02a242)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:38.887685 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb461303-0a60-438e-97e5-134eb283cf38 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:38.889962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 190/379] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:38 2022] GET /identity/v3/projects/d5e32bdaa14a4d31baa44b436b02a242 => generated 344 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:38.890968 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb461303-0a60-438e-97e5-134eb283cf38 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:38.930396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939a394-f30d-4f01-ae33-75f361bb4e98 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:38.969796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5dfd4-92bd-4de1-bb29-6ed7d317dad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:38.970598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5dfd4-92bd-4de1-bb29-6ed7d317dad7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:38.971384 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5dfd4-92bd-4de1-bb29-6ed7d317dad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:38.971736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5dfd4-92bd-4de1-bb29-6ed7d317dad7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:38.972913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5dfd4-92bd-4de1-bb29-6ed7d317dad7 None admin] RBAC: Adding query filter params (group.id=a7c4a8842bf64b77a6ed9d6b706b3c62, role.id=05ca4843c34e463c94de09f756d95318, scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:38.973291 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5dfd4-92bd-4de1-bb29-6ed7d317dad7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:38.974823 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5dfd4-92bd-4de1-bb29-6ed7d317dad7 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:38.983918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 190/380] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jul 20 08:59:38 2022] GET /identity/v3/role_assignments?group.id=a7c4a8842bf64b77a6ed9d6b706b3c62&scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7&role.id=05ca4843c34e463c94de09f756d95318 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:38.984955 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5dfd4-92bd-4de1-bb29-6ed7d317dad7 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:39.041188 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb461303-0a60-438e-97e5-134eb283cf38 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:39.071090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:39.110831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f088c84-5524-4bdd-9e02-2280002bb21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:39.111702 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f088c84-5524-4bdd-9e02-2280002bb21a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:39.111996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f088c84-5524-4bdd-9e02-2280002bb21a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:39.112274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f088c84-5524-4bdd-9e02-2280002bb21a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:39.136695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f088c84-5524-4bdd-9e02-2280002bb21a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:39.145797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-2f088c84-5524-4bdd-9e02-2280002bb21a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:39.145797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:39.145797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:39.145797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:39.145797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:39.145797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:39.145797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:39.145797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:39.145797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:39.145797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:39.145797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:39.145797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:39.145797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:39.145797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:39.145797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:39.145797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:39.146824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:39.146824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:39.146824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:39.146824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:39.146824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:39.146824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:39.146824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:39.146824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:39.146824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:39.146824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:39.146824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:39.146824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:39.146824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:39.146824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:39.146824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:39.146824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:39.147656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:39.147656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:39.147656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:39.147656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:39.147656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:39.147656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:39.147656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:39.147656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:39.147656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:39.147656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:39.147656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:39.147656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:39.147656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 191/381] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:39.163767 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f088c84-5524-4bdd-9e02-2280002bb21a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:39.211924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5dfd4-92bd-4de1-bb29-6ed7d317dad7 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:39.241066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:39.265587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9d8dd-3d0e-4300-9698-c82aaec9a4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:39.266465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9d8dd-3d0e-4300-9698-c82aaec9a4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:39.266776 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9d8dd-3d0e-4300-9698-c82aaec9a4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:39.267056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9d8dd-3d0e-4300-9698-c82aaec9a4e9 None admin] PATH_INFO: `/v3/groups/3b88634c930c4fedbf0bab9055ee9ebe` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:39.274911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9d8dd-3d0e-4300-9698-c82aaec9a4e9 None admin] RBAC: Authorizing `identity:get_group(group_id=3b88634c930c4fedbf0bab9055ee9ebe)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:39.276562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9d8dd-3d0e-4300-9698-c82aaec9a4e9 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:39.278814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 191/382] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jul 20 08:59:38 2022] GET /identity/v3/groups/3b88634c930c4fedbf0bab9055ee9ebe => generated 229 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:39.290855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9d8dd-3d0e-4300-9698-c82aaec9a4e9 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:39.345617 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f088c84-5524-4bdd-9e02-2280002bb21a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:39.409153 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f60f55-45ee-4807-aa09-7c2bc2b9991b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:39.410241 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f60f55-45ee-4807-aa09-7c2bc2b9991b None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:39.410697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f60f55-45ee-4807-aa09-7c2bc2b9991b None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:39.411119 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f60f55-45ee-4807-aa09-7c2bc2b9991b None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:39.412012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f60f55-45ee-4807-aa09-7c2bc2b9991b None admin] RBAC: Adding query filter params (name=member) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:39.412480 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f60f55-45ee-4807-aa09-7c2bc2b9991b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:39.413998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f60f55-45ee-4807-aa09-7c2bc2b9991b None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:39.427904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 192/383] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:39.430008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce088b1-bb15-4a12-b5a5-026584ce0626 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:39.430456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce088b1-bb15-4a12-b5a5-026584ce0626 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:39.431168 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce088b1-bb15-4a12-b5a5-026584ce0626 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:39.433491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 193/384] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:39.434359 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce088b1-bb15-4a12-b5a5-026584ce0626 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:39.442534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:39.451638 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9d8dd-3d0e-4300-9698-c82aaec9a4e9 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:39.493227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f965ea2-cb70-4ccd-af59-17c62b3a3781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:39.493882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f965ea2-cb70-4ccd-af59-17c62b3a3781 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:39.494163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f965ea2-cb70-4ccd-af59-17c62b3a3781 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:39.494440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f965ea2-cb70-4ccd-af59-17c62b3a3781 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:39.495302 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f965ea2-cb70-4ccd-af59-17c62b3a3781 None admin] RBAC: Adding query filter params (group.id=3b88634c930c4fedbf0bab9055ee9ebe, role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=d5e32bdaa14a4d31baa44b436b02a242) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:39.495611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f965ea2-cb70-4ccd-af59-17c62b3a3781 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:39.497050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f965ea2-cb70-4ccd-af59-17c62b3a3781 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:39.506720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 192/385] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jul 20 08:59:39 2022] GET /identity/v3/role_assignments?group.id=3b88634c930c4fedbf0bab9055ee9ebe&scope.project.id=d5e32bdaa14a4d31baa44b436b02a242&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:39.506720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bb0a6-d71f-48ca-9a42-9d77f7f6e8b2 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:39.506720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bb0a6-d71f-48ca-9a42-9d77f7f6e8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:39.506720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bb0a6-d71f-48ca-9a42-9d77f7f6e8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:39.524678 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-765bb0a6-d71f-48ca-9a42-9d77f7f6e8b2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:39.525978 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-765bb0a6-d71f-48ca-9a42-9d77f7f6e8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:39.613736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce088b1-bb15-4a12-b5a5-026584ce0626 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:39.615987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:39.670447 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3e1f1-90f3-4aeb-ae5d-315ff0f9365c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:39.672695 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 193/386] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:39.673933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3e1f1-90f3-4aeb-ae5d-315ff0f9365c None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:39.674397 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3e1f1-90f3-4aeb-ae5d-315ff0f9365c None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:39.674889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3e1f1-90f3-4aeb-ae5d-315ff0f9365c None admin] PATH_INFO: `/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:39.676406 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3e1f1-90f3-4aeb-ae5d-315ff0f9365c None admin] RBAC: Authorizing `identity:get_group(group_id=a7c4a8842bf64b77a6ed9d6b706b3c62)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:39.678153 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3e1f1-90f3-4aeb-ae5d-315ff0f9365c None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:39.679979 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 194/387] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jul 20 08:59:39 2022] GET /identity/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:39.685557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765bb0a6-d71f-48ca-9a42-9d77f7f6e8b2 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:39.718971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3e1f1-90f3-4aeb-ae5d-315ff0f9365c None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:39.833602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:39.846031 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765bb0a6-d71f-48ca-9a42-9d77f7f6e8b2 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:39.867350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3e1f1-90f3-4aeb-ae5d-315ff0f9365c None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:39.887231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f973deb-71d9-4fbd-a3e9-f9c47aadd664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:39.888402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f973deb-71d9-4fbd-a3e9-f9c47aadd664 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:39.889011 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f973deb-71d9-4fbd-a3e9-f9c47aadd664 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:39.891350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f973deb-71d9-4fbd-a3e9-f9c47aadd664 None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:39.893133 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f973deb-71d9-4fbd-a3e9-f9c47aadd664 None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:39.894665 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f973deb-71d9-4fbd-a3e9-f9c47aadd664 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:39.897663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 194/388] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:39 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:39.903755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f973deb-71d9-4fbd-a3e9-f9c47aadd664 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:39.929201 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10728d56-49b0-4796-95e1-7b29977cef72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:39.930411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10728d56-49b0-4796-95e1-7b29977cef72 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:39.930920 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10728d56-49b0-4796-95e1-7b29977cef72 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:39.931404 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10728d56-49b0-4796-95e1-7b29977cef72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:39.946351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10728d56-49b0-4796-95e1-7b29977cef72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:39.955900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-10728d56-49b0-4796-95e1-7b29977cef72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:39.955900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:39.955900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:39.955900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:39.955900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:39.955900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:39.955900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:39.955900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:39.955900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:39.955900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:39.955900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:39.955900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:39.955900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:39.955900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:39.955900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:39.955900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:39.956798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:39.956798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:39.956798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:39.956798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:39.956798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:39.956798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:39.956798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:39.956798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:39.956798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:39.956798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:39.956798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:39.956798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:39.956798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:39.956798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:39.956798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:39.956798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:39.957591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:39.957591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:39.957591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:39.957591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:39.957591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:39.957591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:39.957591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:39.957591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:39.957591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:39.957591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:39.957591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:39.957591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:39.957591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 195/389] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 08:59:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:39.958840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45e752-4509-4ba5-acdf-60a989f4e796 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:39.959050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45e752-4509-4ba5-acdf-60a989f4e796 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:39.959297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45e752-4509-4ba5-acdf-60a989f4e796 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:39.960655 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 196/390] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:39.966085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d288fc-2c0d-4a90-8fb8-705fd1b6c30e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:39.966370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d288fc-2c0d-4a90-8fb8-705fd1b6c30e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:39.966704 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d288fc-2c0d-4a90-8fb8-705fd1b6c30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:39.985698 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-25d288fc-2c0d-4a90-8fb8-705fd1b6c30e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:39.986954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-25d288fc-2c0d-4a90-8fb8-705fd1b6c30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:40.013257 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:40.034916 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f973deb-71d9-4fbd-a3e9-f9c47aadd664 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:40.075603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581af81f-16bc-419e-9a7b-6074cc59c306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:40.076498 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581af81f-16bc-419e-9a7b-6074cc59c306 None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:40.077430 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581af81f-16bc-419e-9a7b-6074cc59c306 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:40.077858 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581af81f-16bc-419e-9a7b-6074cc59c306 None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa/groups/a7c4a8842bf64b77a6ed9d6b706b3c62/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:40.078787 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 197/391] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:40.079728 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d288fc-2c0d-4a90-8fb8-705fd1b6c30e None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:40.081171 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581af81f-16bc-419e-9a7b-6074cc59c306 None admin] RBAC: Authorizing `identity:create_grant(project_id=81fa154d1fc04266abee9f0051cba2fa, group_id=a7c4a8842bf64b77a6ed9d6b706b3c62, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:40.082625 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581af81f-16bc-419e-9a7b-6074cc59c306 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:40.101639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 195/392] 10.222.0.44 () {58 vars in 1442 bytes} [Wed Jul 20 08:59:39 2022] PUT /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa/groups/a7c4a8842bf64b77a6ed9d6b706b3c62/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:59:40.140664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581af81f-16bc-419e-9a7b-6074cc59c306 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:40.168180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:40.209164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d288fc-2c0d-4a90-8fb8-705fd1b6c30e None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:40.252625 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a903649-8d1a-48bb-9798-21a7624a9c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:40.253418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a903649-8d1a-48bb-9798-21a7624a9c2d None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:40.253704 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a903649-8d1a-48bb-9798-21a7624a9c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:40.253984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a903649-8d1a-48bb-9798-21a7624a9c2d None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:40.254769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a903649-8d1a-48bb-9798-21a7624a9c2d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:40.255313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a903649-8d1a-48bb-9798-21a7624a9c2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:40.256608 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a903649-8d1a-48bb-9798-21a7624a9c2d None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:40.268054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 198/393] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:40.277477 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581af81f-16bc-419e-9a7b-6074cc59c306 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:40.288562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a903649-8d1a-48bb-9798-21a7624a9c2d None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:40.316830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84de154-d9d7-493f-bf45-29f8ed609def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:40.317648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84de154-d9d7-493f-bf45-29f8ed609def None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:40.317939 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84de154-d9d7-493f-bf45-29f8ed609def None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:40.318222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84de154-d9d7-493f-bf45-29f8ed609def None admin] PATH_INFO: `/v3/roles/member` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:40.343723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84de154-d9d7-493f-bf45-29f8ed609def None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:40.347230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:40.354877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-f84de154-d9d7-493f-bf45-29f8ed609def None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:40.354877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:40.354877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:40.354877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:40.354877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:40.354877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:40.354877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:40.354877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:40.354877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:40.354877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:40.354877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:40.354877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:40.354877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:40.354877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:40.354877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:40.354877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:40.355814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:40.355814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:40.355814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:40.355814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:40.355814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:40.355814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:40.355814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:40.355814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:40.355814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:40.355814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:40.355814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:40.355814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:40.355814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:40.355814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:40.355814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:40.355814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:40.357159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:40.357159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:40.357159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:40.357159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:40.357159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:40.357159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:40.357159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:40.357159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:40.357159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:40.357159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:40.357159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:40.357159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:40.357159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 196/394] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:40.367099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84de154-d9d7-493f-bf45-29f8ed609def None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:40.482762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a903649-8d1a-48bb-9798-21a7624a9c2d None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:40.503682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:40.531671 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84de154-d9d7-493f-bf45-29f8ed609def None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:40.542220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206ca40-93bd-4278-b2c2-e494052b2e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:40.543247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206ca40-93bd-4278-b2c2-e494052b2e85 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:40.543453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206ca40-93bd-4278-b2c2-e494052b2e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:40.543724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206ca40-93bd-4278-b2c2-e494052b2e85 None admin] PATH_INFO: `/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:40.547795 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206ca40-93bd-4278-b2c2-e494052b2e85 None admin] RBAC: Authorizing `identity:get_group(group_id=a7c4a8842bf64b77a6ed9d6b706b3c62)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:40.548934 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206ca40-93bd-4278-b2c2-e494052b2e85 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:40.551886 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 199/395] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jul 20 08:59:40 2022] GET /identity/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62 => generated 226 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:40.556272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206ca40-93bd-4278-b2c2-e494052b2e85 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:40.585622 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a397b-2a33-44f8-a73d-4f41af230876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:40.586656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a397b-2a33-44f8-a73d-4f41af230876 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:40.587170 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a397b-2a33-44f8-a73d-4f41af230876 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:40.587751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a397b-2a33-44f8-a73d-4f41af230876 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:40.588844 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a397b-2a33-44f8-a73d-4f41af230876 None admin] RBAC: Adding query filter params (name=member) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:40.589407 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a397b-2a33-44f8-a73d-4f41af230876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:40.591168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a397b-2a33-44f8-a73d-4f41af230876 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:40.605229 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 197/396] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:40.611257 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a397b-2a33-44f8-a73d-4f41af230876 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:40.692309 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:40.724800 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206ca40-93bd-4278-b2c2-e494052b2e85 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:40.747289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a397b-2a33-44f8-a73d-4f41af230876 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:40.788699 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99758d86-f8bc-4023-a574-1828261edb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:40.789527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99758d86-f8bc-4023-a574-1828261edb18 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:40.789807 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99758d86-f8bc-4023-a574-1828261edb18 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:40.790077 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99758d86-f8bc-4023-a574-1828261edb18 None admin] PATH_INFO: `/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:40.794088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99758d86-f8bc-4023-a574-1828261edb18 None admin] RBAC: Authorizing `identity:get_group(group_id=a7c4a8842bf64b77a6ed9d6b706b3c62)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:40.796102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99758d86-f8bc-4023-a574-1828261edb18 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:40.797737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 198/397] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jul 20 08:59:40 2022] GET /identity/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:40.804911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede0297-a98e-4b4e-bfb8-362c1783d636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:40.805797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede0297-a98e-4b4e-bfb8-362c1783d636 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:40.806079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede0297-a98e-4b4e-bfb8-362c1783d636 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:40.806355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede0297-a98e-4b4e-bfb8-362c1783d636 None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:40.807240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99758d86-f8bc-4023-a574-1828261edb18 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:40.807806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede0297-a98e-4b4e-bfb8-362c1783d636 None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:40.809237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede0297-a98e-4b4e-bfb8-362c1783d636 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:40.811736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 200/398] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:40 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:40.817095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede0297-a98e-4b4e-bfb8-362c1783d636 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:40.831713 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:41.017298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99758d86-f8bc-4023-a574-1828261edb18 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:41.037203 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede0297-a98e-4b4e-bfb8-362c1783d636 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:41.096001 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66fdd3-c4f1-4767-a6b7-cd14d1ff98ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:41.098394 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66fdd3-c4f1-4767-a6b7-cd14d1ff98ae None admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:41.099321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66fdd3-c4f1-4767-a6b7-cd14d1ff98ae None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:41.100295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66fdd3-c4f1-4767-a6b7-cd14d1ff98ae None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa/groups/a7c4a8842bf64b77a6ed9d6b706b3c62/roles/05ca4843c34e463c94de09f756d95318` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:41.104271 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66fdd3-c4f1-4767-a6b7-cd14d1ff98ae None admin] RBAC: Authorizing `identity:create_grant(project_id=81fa154d1fc04266abee9f0051cba2fa, group_id=a7c4a8842bf64b77a6ed9d6b706b3c62, role_id=05ca4843c34e463c94de09f756d95318)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:41.105091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a808ae-1942-454b-81c6-16689d51539c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:41.106186 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a808ae-1942-454b-81c6-16689d51539c None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:41.108417 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66fdd3-c4f1-4767-a6b7-cd14d1ff98ae None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:41.109189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a808ae-1942-454b-81c6-16689d51539c None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:41.109768 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a808ae-1942-454b-81c6-16689d51539c None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:41.113442 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a808ae-1942-454b-81c6-16689d51539c None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:41.115163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a808ae-1942-454b-81c6-16689d51539c None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:41.119803 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 199/399] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:40 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:41.126551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a808ae-1942-454b-81c6-16689d51539c None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:41.145600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 201/400] 10.222.0.44 () {58 vars in 1442 bytes} [Wed Jul 20 08:59:40 2022] PUT /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa/groups/a7c4a8842bf64b77a6ed9d6b706b3c62/roles/05ca4843c34e463c94de09f756d95318 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:59:41.234471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:41.266844 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a808ae-1942-454b-81c6-16689d51539c None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:41.314784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e2861-4444-4ae1-920d-e0890a3ed3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:41.315578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e2861-4444-4ae1-920d-e0890a3ed3fa None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:41.316000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e2861-4444-4ae1-920d-e0890a3ed3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:41.316279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e2861-4444-4ae1-920d-e0890a3ed3fa None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7/groups/a7c4a8842bf64b77a6ed9d6b706b3c62/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:41.318307 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e2861-4444-4ae1-920d-e0890a3ed3fa None admin] RBAC: Authorizing `identity:create_grant(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7, group_id=a7c4a8842bf64b77a6ed9d6b706b3c62, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:41.320114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e2861-4444-4ae1-920d-e0890a3ed3fa None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:41.340665 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 200/401] 10.222.0.44 () {58 vars in 1442 bytes} [Wed Jul 20 08:59:41 2022] PUT /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7/groups/a7c4a8842bf64b77a6ed9d6b706b3c62/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:59:41.676799 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b1435-dc24-47b5-b99e-98361ab5b2a9 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:41.677155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b1435-dc24-47b5-b99e-98361ab5b2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:41.677357 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b1435-dc24-47b5-b99e-98361ab5b2a9 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:41.679121 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 202/402] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:41.684365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf1f3e-5887-46eb-bf33-8be138fde5b4 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:41.684812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf1f3e-5887-46eb-bf33-8be138fde5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:41.684912 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf1f3e-5887-46eb-bf33-8be138fde5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:41.709547 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-efaf1f3e-5887-46eb-bf33-8be138fde5b4 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:41.710832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-efaf1f3e-5887-46eb-bf33-8be138fde5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:41.802585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 201/403] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:41.849013 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b1435-dc24-47b5-b99e-98361ab5b2a9 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:41.869764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:41.980314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b1435-dc24-47b5-b99e-98361ab5b2a9 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:42.023718 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f6ab8-945e-4492-9f71-4b36e23d0d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:42.024537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f6ab8-945e-4492-9f71-4b36e23d0d68 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:42.024828 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f6ab8-945e-4492-9f71-4b36e23d0d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:42.025106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f6ab8-945e-4492-9f71-4b36e23d0d68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:42.040021 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f6ab8-945e-4492-9f71-4b36e23d0d68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:42.048246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-574f6ab8-945e-4492-9f71-4b36e23d0d68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:42.048246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:42.048246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:42.048246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:42.048246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:42.048246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:42.048246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:42.048246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:42.048246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:42.048246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:42.048246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:42.048246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:42.048246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:42.048246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:42.048246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:42.048246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:42.049132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:42.049132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:42.049132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:42.049132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:42.049132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:42.049132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:42.049132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:42.049132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:42.049132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:42.049132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:42.049132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:42.049132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:42.049132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:42.049132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:42.049132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:42.049132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:42.049917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:42.049917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:42.049917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:42.049917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:42.049917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:42.049917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:42.049917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:42.049917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:42.049917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:42.049917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:42.049917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:42.049917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:42.049917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 203/404] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 08:59:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:42.055422 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf1f3e-5887-46eb-bf33-8be138fde5b4 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:42.142643 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:42.144700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765c56e-272c-45fb-9db2-4e5bc402e938 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:42.145190 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765c56e-272c-45fb-9db2-4e5bc402e938 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:42.145279 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765c56e-272c-45fb-9db2-4e5bc402e938 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:42.146635 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 204/405] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:59:42.152174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d99724-1031-480e-b9d2-0e88cb5d4955 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:42.152458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d99724-1031-480e-b9d2-0e88cb5d4955 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:42.152746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d99724-1031-480e-b9d2-0e88cb5d4955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:42.185030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf1f3e-5887-46eb-bf33-8be138fde5b4 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:42.187700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-b4d99724-1031-480e-b9d2-0e88cb5d4955 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:42.188897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b4d99724-1031-480e-b9d2-0e88cb5d4955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:42.228420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95c7ed-2e21-45c3-b7b1-ca29dff099a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:42.230449 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95c7ed-2e21-45c3-b7b1-ca29dff099a8 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:42.231118 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95c7ed-2e21-45c3-b7b1-ca29dff099a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:42.232638 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95c7ed-2e21-45c3-b7b1-ca29dff099a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:42.233631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95c7ed-2e21-45c3-b7b1-ca29dff099a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:42.234310 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95c7ed-2e21-45c3-b7b1-ca29dff099a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:42.236538 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95c7ed-2e21-45c3-b7b1-ca29dff099a8 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:42.250920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 202/406] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:42.257171 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95c7ed-2e21-45c3-b7b1-ca29dff099a8 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:42.319974 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 205/407] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:42.354842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d99724-1031-480e-b9d2-0e88cb5d4955 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:42.366396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:42.430503 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95c7ed-2e21-45c3-b7b1-ca29dff099a8 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:42.491549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:42.504673 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b415ce2-5c77-4c1d-b30a-61f873794fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:42.505961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b415ce2-5c77-4c1d-b30a-61f873794fdb None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:42.507883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b415ce2-5c77-4c1d-b30a-61f873794fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:42.508991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b415ce2-5c77-4c1d-b30a-61f873794fdb None admin] PATH_INFO: `/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:42.510506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b415ce2-5c77-4c1d-b30a-61f873794fdb None admin] RBAC: Authorizing `identity:get_group(group_id=a7c4a8842bf64b77a6ed9d6b706b3c62)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:42.512867 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b415ce2-5c77-4c1d-b30a-61f873794fdb None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:42.515866 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 203/408] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jul 20 08:59:42 2022] GET /identity/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62 => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:42.521804 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b415ce2-5c77-4c1d-b30a-61f873794fdb None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:42.562415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d99724-1031-480e-b9d2-0e88cb5d4955 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:42.635980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a344a03-2b5e-4043-b6fc-ee5645bd719f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:42.636792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a344a03-2b5e-4043-b6fc-ee5645bd719f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:42.637076 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a344a03-2b5e-4043-b6fc-ee5645bd719f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:42.637351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a344a03-2b5e-4043-b6fc-ee5645bd719f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:42.656425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a344a03-2b5e-4043-b6fc-ee5645bd719f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:42.673237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-8a344a03-2b5e-4043-b6fc-ee5645bd719f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:42.673237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:42.673237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:42.673237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:42.673237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:42.673237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:42.673237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:42.673237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:42.673237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:42.673237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:42.673237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:42.673237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:42.673237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:42.673237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:42.673237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:42.673237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:42.674181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:42.674181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:42.674181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:42.674181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:42.674181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:42.674181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:42.674181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:42.674181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:42.674181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:42.674181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:42.674181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:42.674181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:42.674181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:42.674181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:42.674181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:42.674181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:42.675025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:42.675025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:42.675025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:42.675025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:42.675025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:42.675025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:42.675025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:42.675025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:42.675025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:42.675025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:42.675025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:42.675025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:42.677430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 206/409] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 08:59:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:42.684327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a344a03-2b5e-4043-b6fc-ee5645bd719f None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:42.687596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b415ce2-5c77-4c1d-b30a-61f873794fdb None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:42.725226 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3182f34-4db8-4931-b5e4-89104ed82d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:42.726092 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3182f34-4db8-4931-b5e4-89104ed82d38 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:42.726556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3182f34-4db8-4931-b5e4-89104ed82d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:42.727004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3182f34-4db8-4931-b5e4-89104ed82d38 None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:42.728494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3182f34-4db8-4931-b5e4-89104ed82d38 None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:42.730216 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3182f34-4db8-4931-b5e4-89104ed82d38 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:42.732489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 204/410] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:42 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:42.737892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3182f34-4db8-4931-b5e4-89104ed82d38 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:42.826063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:42.879022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:42.894446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a344a03-2b5e-4043-b6fc-ee5645bd719f None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:42.917648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3182f34-4db8-4931-b5e4-89104ed82d38 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:42.940105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f8d52-5036-4332-a965-41e5f2cc434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:42.941073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f8d52-5036-4332-a965-41e5f2cc434a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:42.941504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f8d52-5036-4332-a965-41e5f2cc434a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:42.941963 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f8d52-5036-4332-a965-41e5f2cc434a None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:42.943425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f8d52-5036-4332-a965-41e5f2cc434a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:42.944422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f8d52-5036-4332-a965-41e5f2cc434a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:42.946601 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f8d52-5036-4332-a965-41e5f2cc434a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:42.958469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 207/411] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:42.959957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cd3b0-8a98-4537-96f1-ce5009aee76a None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:42.960378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cd3b0-8a98-4537-96f1-ce5009aee76a None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:42.960822 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cd3b0-8a98-4537-96f1-ce5009aee76a None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:42.962167 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 208/412] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:42.963395 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cd3b0-8a98-4537-96f1-ce5009aee76a None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:42.965179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9713a-1e3d-4b50-9ae7-16ae5abfb1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:42.966268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9713a-1e3d-4b50-9ae7-16ae5abfb1c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:42.970708 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9713a-1e3d-4b50-9ae7-16ae5abfb1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:42.971250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9713a-1e3d-4b50-9ae7-16ae5abfb1c6 None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7/groups/a7c4a8842bf64b77a6ed9d6b706b3c62/roles/05ca4843c34e463c94de09f756d95318` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:42.973910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f9713a-1e3d-4b50-9ae7-16ae5abfb1c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7, group_id=a7c4a8842bf64b77a6ed9d6b706b3c62, role_id=05ca4843c34e463c94de09f756d95318)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:42.975737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f9713a-1e3d-4b50-9ae7-16ae5abfb1c6 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:42.991810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 205/413] 10.222.0.44 () {58 vars in 1442 bytes} [Wed Jul 20 08:59:42 2022] PUT /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7/groups/a7c4a8842bf64b77a6ed9d6b706b3c62/roles/05ca4843c34e463c94de09f756d95318 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:59:42.993542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e7581-9184-42e8-a9ee-73eb07f154a8 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:42.994188 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e7581-9184-42e8-a9ee-73eb07f154a8 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:42.994808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e7581-9184-42e8-a9ee-73eb07f154a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:43.013899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-883e7581-9184-42e8-a9ee-73eb07f154a8 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:43.016230 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-883e7581-9184-42e8-a9ee-73eb07f154a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:43.091028 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cd3b0-8a98-4537-96f1-ce5009aee76a None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:43.096548 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 206/414] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:43.123028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e7581-9184-42e8-a9ee-73eb07f154a8 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:43.124823 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:43.133485 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63717e17-3c8b-4102-9fba-e5cb40453b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:43.134931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63717e17-3c8b-4102-9fba-e5cb40453b20 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:43.135427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63717e17-3c8b-4102-9fba-e5cb40453b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:43.136242 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63717e17-3c8b-4102-9fba-e5cb40453b20 None admin] PATH_INFO: `/v3/groups/3b88634c930c4fedbf0bab9055ee9ebe` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:43.137958 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63717e17-3c8b-4102-9fba-e5cb40453b20 None admin] RBAC: Authorizing `identity:get_group(group_id=3b88634c930c4fedbf0bab9055ee9ebe)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:43.140454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63717e17-3c8b-4102-9fba-e5cb40453b20 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:43.142596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 209/415] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jul 20 08:59:42 2022] GET /identity/v3/groups/3b88634c930c4fedbf0bab9055ee9ebe => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:43.148593 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63717e17-3c8b-4102-9fba-e5cb40453b20 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:43.231350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:43.300532 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63717e17-3c8b-4102-9fba-e5cb40453b20 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:43.317875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e7581-9184-42e8-a9ee-73eb07f154a8 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:43.345098 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd22a2-fc70-406d-8e55-d121fb14450d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:43.346511 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd22a2-fc70-406d-8e55-d121fb14450d None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:43.347322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd22a2-fc70-406d-8e55-d121fb14450d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:43.347805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd22a2-fc70-406d-8e55-d121fb14450d None admin] PATH_INFO: `/v3/projects/d5e32bdaa14a4d31baa44b436b02a242` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:43.349630 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd22a2-fc70-406d-8e55-d121fb14450d None admin] RBAC: Authorizing `identity:get_project(project_id=d5e32bdaa14a4d31baa44b436b02a242)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:43.351911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd22a2-fc70-406d-8e55-d121fb14450d None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:43.354863 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 210/416] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:43 2022] GET /identity/v3/projects/d5e32bdaa14a4d31baa44b436b02a242 => generated 344 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:43.360283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd22a2-fc70-406d-8e55-d121fb14450d None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:43.364484 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7325e3-75ad-4060-a910-72beb5f5240b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:43.366210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7325e3-75ad-4060-a910-72beb5f5240b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:43.366504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7325e3-75ad-4060-a910-72beb5f5240b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:43.366819 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7325e3-75ad-4060-a910-72beb5f5240b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:43.381695 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7325e3-75ad-4060-a910-72beb5f5240b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:43.391295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-ab7325e3-75ad-4060-a910-72beb5f5240b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:43.391295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:43.391295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:43.391295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:43.391295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:43.391295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:43.391295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:43.391295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:43.391295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:43.391295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:43.391295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:43.391295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:43.391295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:43.391295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:43.391295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:43.391295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:43.392207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:43.392207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:43.392207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:43.392207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:43.392207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:43.392207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:43.392207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:43.392207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:43.392207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:43.392207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:43.392207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:43.392207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:43.392207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:43.392207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:43.392207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:43.392207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:43.392987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:43.392987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:43.392987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:43.392987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:43.392987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:43.392987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:43.392987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:43.392987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:43.392987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:43.392987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:43.392987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:43.392987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:43.395869 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 207/417] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:43.401522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7325e3-75ad-4060-a910-72beb5f5240b None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:43.448870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:43.490176 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd22a2-fc70-406d-8e55-d121fb14450d None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:43.512355 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7325e3-75ad-4060-a910-72beb5f5240b None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:43.527536 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:43.529162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219ecc7-25b0-4773-a84b-f484e0300e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:43.531680 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219ecc7-25b0-4773-a84b-f484e0300e62 None admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:43.532221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219ecc7-25b0-4773-a84b-f484e0300e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:43.533230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219ecc7-25b0-4773-a84b-f484e0300e62 None admin] PATH_INFO: `/v3/projects/d5e32bdaa14a4d31baa44b436b02a242/groups/3b88634c930c4fedbf0bab9055ee9ebe/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:43.543164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219ecc7-25b0-4773-a84b-f484e0300e62 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5e32bdaa14a4d31baa44b436b02a242, group_id=3b88634c930c4fedbf0bab9055ee9ebe, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:43.545011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3219ecc7-25b0-4773-a84b-f484e0300e62 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:43.569284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 211/418] 10.222.0.44 () {58 vars in 1442 bytes} [Wed Jul 20 08:59:43 2022] PUT /identity/v3/projects/d5e32bdaa14a4d31baa44b436b02a242/groups/3b88634c930c4fedbf0bab9055ee9ebe/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:59:43.600950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80379b-e95c-44de-8321-37950824c777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:43.601654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80379b-e95c-44de-8321-37950824c777 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:43.601946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80379b-e95c-44de-8321-37950824c777 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:43.602225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80379b-e95c-44de-8321-37950824c777 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:43.603036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80379b-e95c-44de-8321-37950824c777 None admin] RBAC: Adding query filter params (name=member) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:43.603347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80379b-e95c-44de-8321-37950824c777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:43.604776 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80379b-e95c-44de-8321-37950824c777 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:43.622662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 208/419] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:43.630300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219ecc7-25b0-4773-a84b-f484e0300e62 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:43.797752 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219ecc7-25b0-4773-a84b-f484e0300e62 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:43.840458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:43.850562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3051a-bd2a-4e78-b5be-cccb6d934775 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:43.851143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3051a-bd2a-4e78-b5be-cccb6d934775 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:43.851143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3051a-bd2a-4e78-b5be-cccb6d934775 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:43.852660 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 209/420] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:43.857796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa49065-638f-4559-94bf-1c05640697cd None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:43.858706 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa49065-638f-4559-94bf-1c05640697cd None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:43.859034 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa49065-638f-4559-94bf-1c05640697cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:43.860979 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2395f-fc22-4d53-8ef8-832f6bd67415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:43.861646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2395f-fc22-4d53-8ef8-832f6bd67415 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:43.861925 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2395f-fc22-4d53-8ef8-832f6bd67415 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:43.862202 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c2395f-fc22-4d53-8ef8-832f6bd67415 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:43.895705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-caa49065-638f-4559-94bf-1c05640697cd None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:43.896912 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-caa49065-638f-4559-94bf-1c05640697cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:43.947496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2395f-fc22-4d53-8ef8-832f6bd67415 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:43.948952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c2395f-fc22-4d53-8ef8-832f6bd67415 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:44.016532 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 210/421] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:44.017953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33490ed9-a12e-462b-8643-eb0a76d26b6f None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:44.018249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33490ed9-a12e-462b-8643-eb0a76d26b6f None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:44.018533 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33490ed9-a12e-462b-8643-eb0a76d26b6f None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:44.019890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 211/422] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:44.040750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dc442-4ba8-4ca4-9be0-17c2db2667c7 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:44.041212 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dc442-4ba8-4ca4-9be0-17c2db2667c7 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:44.041587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dc442-4ba8-4ca4-9be0-17c2db2667c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:44.063844 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-725dc442-4ba8-4ca4-9be0-17c2db2667c7 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:44.065067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-725dc442-4ba8-4ca4-9be0-17c2db2667c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:44.092834 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 212/423] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:44.098255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2395f-fc22-4d53-8ef8-832f6bd67415 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:44.130327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:44.157897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 212/424] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:44.159214 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dc442-4ba8-4ca4-9be0-17c2db2667c7 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:44.196804 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:44.231548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c2395f-fc22-4d53-8ef8-832f6bd67415 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:44.267769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a5e3f-dbe6-4432-b324-6da307d1b423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:44.268568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a5e3f-dbe6-4432-b324-6da307d1b423 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:44.268852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a5e3f-dbe6-4432-b324-6da307d1b423 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:44.269131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a5e3f-dbe6-4432-b324-6da307d1b423 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:44.273138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dc442-4ba8-4ca4-9be0-17c2db2667c7 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:44.279474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a5e3f-dbe6-4432-b324-6da307d1b423 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:44.287798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-f43a5e3f-dbe6-4432-b324-6da307d1b423 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:44.287798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:44.287798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:44.287798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:44.287798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:44.287798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:44.287798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:44.287798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:44.287798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:44.287798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:44.287798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:44.287798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:44.287798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:44.287798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:44.287798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:44.287798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:44.288674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:44.288674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:44.288674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:44.288674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:44.288674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:44.288674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:44.288674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:44.288674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:44.288674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:44.288674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:44.288674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:44.288674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:44.288674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:44.288674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:44.288674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:44.288674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:44.289482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:44.289482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:44.289482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:44.289482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:44.289482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:44.289482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:44.289482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:44.289482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:44.289482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:44.289482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:44.289482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:59:44.289482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:44.293759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 213/425] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:44.294600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a5e3f-dbe6-4432-b324-6da307d1b423 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:44.313618 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021b31e-7ad1-4927-8a81-91bfccab9d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:44.314942 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021b31e-7ad1-4927-8a81-91bfccab9d69 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:44.315530 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021b31e-7ad1-4927-8a81-91bfccab9d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:44.316026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021b31e-7ad1-4927-8a81-91bfccab9d69 None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:44.317494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021b31e-7ad1-4927-8a81-91bfccab9d69 None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:44.319537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021b31e-7ad1-4927-8a81-91bfccab9d69 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:44.321754 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 213/426] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:44 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:44.327832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021b31e-7ad1-4927-8a81-91bfccab9d69 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:44.384584 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:44.400420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a5e3f-dbe6-4432-b324-6da307d1b423 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:44.437366 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021b31e-7ad1-4927-8a81-91bfccab9d69 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:44.439534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208abb79-a5a8-48c3-b871-2cee159fb142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:44.440980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208abb79-a5a8-48c3-b871-2cee159fb142 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:44.440980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208abb79-a5a8-48c3-b871-2cee159fb142 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:44.440980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208abb79-a5a8-48c3-b871-2cee159fb142 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:44.442999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:44.452529 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208abb79-a5a8-48c3-b871-2cee159fb142 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:44.462990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-208abb79-a5a8-48c3-b871-2cee159fb142 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:44.462990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:44.462990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:44.462990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:44.462990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:44.462990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:44.462990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:44.462990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:44.462990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:44.462990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:44.462990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:44.462990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:44.462990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:44.462990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:44.462990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:44.462990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:44.463881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:44.463881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:44.463881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:44.463881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:44.463881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:44.463881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:44.463881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:44.463881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:44.463881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:44.463881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:44.463881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:44.463881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:44.463881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:44.463881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:44.463881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:44.463881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:44.464690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:44.464690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:44.464690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:44.464690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:44.464690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:44.464690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:44.464690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:44.464690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:44.464690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:44.464690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:44.464690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:44.464690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:44.470773 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 214/427] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 08:59:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:44.471681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208abb79-a5a8-48c3-b871-2cee159fb142 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:44.480672 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adeddc3-dd7c-43d4-85b6-f0edae0e4a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:44.481906 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adeddc3-dd7c-43d4-85b6-f0edae0e4a97 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:44.482409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adeddc3-dd7c-43d4-85b6-f0edae0e4a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:44.483498 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adeddc3-dd7c-43d4-85b6-f0edae0e4a97 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:44.484850 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adeddc3-dd7c-43d4-85b6-f0edae0e4a97 None admin] RBAC: Adding query filter params (name=member) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:44.485394 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adeddc3-dd7c-43d4-85b6-f0edae0e4a97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:44.492447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adeddc3-dd7c-43d4-85b6-f0edae0e4a97 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:44.503526 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 214/428] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 08:59:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:44.504668 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adeddc3-dd7c-43d4-85b6-f0edae0e4a97 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:44.587320 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208abb79-a5a8-48c3-b871-2cee159fb142 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:44.622296 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c22674-915e-4fe0-91b9-713d524d8b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:44.622703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adeddc3-dd7c-43d4-85b6-f0edae0e4a97 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:44.623215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c22674-915e-4fe0-91b9-713d524d8b3a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:44.624072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c22674-915e-4fe0-91b9-713d524d8b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:44.624347 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c22674-915e-4fe0-91b9-713d524d8b3a None admin] PATH_INFO: `/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:44.625794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c22674-915e-4fe0-91b9-713d524d8b3a None admin] RBAC: Authorizing `identity:get_group(group_id=a7c4a8842bf64b77a6ed9d6b706b3c62)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:44.627459 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c22674-915e-4fe0-91b9-713d524d8b3a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:44.629422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 215/429] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jul 20 08:59:44 2022] GET /identity/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:44.630255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c22674-915e-4fe0-91b9-713d524d8b3a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:44.648738 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:44.675033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990585a-511a-4b99-b2b4-7ee136eee639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:44.675709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990585a-511a-4b99-b2b4-7ee136eee639 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:44.675991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990585a-511a-4b99-b2b4-7ee136eee639 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:44.676270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990585a-511a-4b99-b2b4-7ee136eee639 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:44.677018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a990585a-511a-4b99-b2b4-7ee136eee639 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:44.677321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a990585a-511a-4b99-b2b4-7ee136eee639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:44.678701 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a990585a-511a-4b99-b2b4-7ee136eee639 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:44.689787 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 215/430] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:44.695910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990585a-511a-4b99-b2b4-7ee136eee639 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:44.732479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:44.760204 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c22674-915e-4fe0-91b9-713d524d8b3a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:44.803633 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b08ad1-1d35-4066-b76e-9095beec9c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:44.804286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b08ad1-1d35-4066-b76e-9095beec9c2a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:44.804567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b08ad1-1d35-4066-b76e-9095beec9c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:44.804843 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b08ad1-1d35-4066-b76e-9095beec9c2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:44.831274 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990585a-511a-4b99-b2b4-7ee136eee639 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:44.877552 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431eabc8-9d71-4841-91be-e6287576ea18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:44.878303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431eabc8-9d71-4841-91be-e6287576ea18 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:44.878542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431eabc8-9d71-4841-91be-e6287576ea18 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:44.878870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431eabc8-9d71-4841-91be-e6287576ea18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:44.879754 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431eabc8-9d71-4841-91be-e6287576ea18 None admin] RBAC: Adding query filter params (group.id=a7c4a8842bf64b77a6ed9d6b706b3c62, role.id=0e2011aac3e1430cb96bdbf810e89eb7, scope.project.id=81fa154d1fc04266abee9f0051cba2fa) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:44.880068 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431eabc8-9d71-4841-91be-e6287576ea18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:44.882001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431eabc8-9d71-4841-91be-e6287576ea18 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:44.890140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 216/431] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jul 20 08:59:44 2022] GET /identity/v3/role_assignments?group.id=a7c4a8842bf64b77a6ed9d6b706b3c62&scope.project.id=81fa154d1fc04266abee9f0051cba2fa&role.id=0e2011aac3e1430cb96bdbf810e89eb7 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:44.891254 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431eabc8-9d71-4841-91be-e6287576ea18 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:44.901402 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b08ad1-1d35-4066-b76e-9095beec9c2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:44.904273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b08ad1-1d35-4066-b76e-9095beec9c2a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:44.924039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:45.007763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:45.019739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431eabc8-9d71-4841-91be-e6287576ea18 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:45.026729 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 216/432] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:45.028182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b08ad1-1d35-4066-b76e-9095beec9c2a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:45.066892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c50b7e-9bd9-468f-b954-2f0f5f36c749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:45.067982 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c50b7e-9bd9-468f-b954-2f0f5f36c749 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:45.068468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c50b7e-9bd9-468f-b954-2f0f5f36c749 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:45.069263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c50b7e-9bd9-468f-b954-2f0f5f36c749 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:45.131953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b08ad1-1d35-4066-b76e-9095beec9c2a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:45.147832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:45.179554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a2e17-e65f-480b-b383-1e385d4fedf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:45.180362 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a2e17-e65f-480b-b383-1e385d4fedf5 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:45.180649 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a2e17-e65f-480b-b383-1e385d4fedf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:45.180930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a2e17-e65f-480b-b383-1e385d4fedf5 None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:45.185790 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c50b7e-9bd9-468f-b954-2f0f5f36c749 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:45.187499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a2e17-e65f-480b-b383-1e385d4fedf5 None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:45.188524 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c50b7e-9bd9-468f-b954-2f0f5f36c749 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:45.188944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a2e17-e65f-480b-b383-1e385d4fedf5 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:45.190748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 217/433] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:45 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:45.196548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a2e17-e65f-480b-b383-1e385d4fedf5 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:45.300979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 217/434] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:45.305733 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:45.307069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c50b7e-9bd9-468f-b954-2f0f5f36c749 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:45.308725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a2e17-e65f-480b-b383-1e385d4fedf5 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:45.346008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34508e2e-0640-4972-88fc-1edf05c0fac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:45.347669 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34508e2e-0640-4972-88fc-1edf05c0fac0 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:45.347669 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34508e2e-0640-4972-88fc-1edf05c0fac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:45.347669 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34508e2e-0640-4972-88fc-1edf05c0fac0 None admin] PATH_INFO: `/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:45.348561 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34508e2e-0640-4972-88fc-1edf05c0fac0 None admin] RBAC: Authorizing `identity:get_group(group_id=a7c4a8842bf64b77a6ed9d6b706b3c62)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:45.349993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34508e2e-0640-4972-88fc-1edf05c0fac0 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:45.351988 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 218/435] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jul 20 08:59:45 2022] GET /identity/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:45.358310 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34508e2e-0640-4972-88fc-1edf05c0fac0 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:45.413743 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:45.424057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c50b7e-9bd9-468f-b954-2f0f5f36c749 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:45.462215 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6658df-dde0-449e-9c2d-7599677a66d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:45.463339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6658df-dde0-449e-9c2d-7599677a66d9 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:45.463768 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6658df-dde0-449e-9c2d-7599677a66d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:45.464228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6658df-dde0-449e-9c2d-7599677a66d9 None admin] PATH_INFO: `/v3/projects/81fa154d1fc04266abee9f0051cba2fa` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:45.465603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6658df-dde0-449e-9c2d-7599677a66d9 None admin] RBAC: Authorizing `identity:get_project(project_id=81fa154d1fc04266abee9f0051cba2fa)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:45.467288 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6658df-dde0-449e-9c2d-7599677a66d9 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:45.469465 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34508e2e-0640-4972-88fc-1edf05c0fac0 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:45.469909 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 218/436] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:45 2022] GET /identity/v3/projects/81fa154d1fc04266abee9f0051cba2fa => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:45.475465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6658df-dde0-449e-9c2d-7599677a66d9 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:45.508298 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb4b1f-d890-404e-b45e-2fc8bf3164f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:45.508986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb4b1f-d890-404e-b45e-2fc8bf3164f6 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:45.509267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb4b1f-d890-404e-b45e-2fc8bf3164f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:45.509529 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb4b1f-d890-404e-b45e-2fc8bf3164f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:45.510638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb4b1f-d890-404e-b45e-2fc8bf3164f6 None admin] RBAC: Adding query filter params (group.id=a7c4a8842bf64b77a6ed9d6b706b3c62, role.id=0e2011aac3e1430cb96bdbf810e89eb7, scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:45.511044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb4b1f-d890-404e-b45e-2fc8bf3164f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:45.512595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb4b1f-d890-404e-b45e-2fc8bf3164f6 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:45.520407 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 219/437] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jul 20 08:59:45 2022] GET /identity/v3/role_assignments?group.id=a7c4a8842bf64b77a6ed9d6b706b3c62&scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7&role.id=0e2011aac3e1430cb96bdbf810e89eb7 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:45.563370 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:45.598828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6658df-dde0-449e-9c2d-7599677a66d9 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:45.614513 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f14c51-604e-4403-8ec1-600bb488b0e4 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:45.614876 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f14c51-604e-4403-8ec1-600bb488b0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:45.615126 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f14c51-604e-4403-8ec1-600bb488b0e4 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:45.616928 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 220/438] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:59:45.623664 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5f880-9203-461b-b32c-690dfa83e82e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:45.623993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5f880-9203-461b-b32c-690dfa83e82e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:45.624237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5f880-9203-461b-b32c-690dfa83e82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:45.643312 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb39a43-9c7d-4ed6-a7d2-cfdccce5ecb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:45.644645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb39a43-9c7d-4ed6-a7d2-cfdccce5ecb0 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:45.645138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb39a43-9c7d-4ed6-a7d2-cfdccce5ecb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:45.645277 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-b6c5f880-9203-461b-b32c-690dfa83e82e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:45.645755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb39a43-9c7d-4ed6-a7d2-cfdccce5ecb0 None admin] PATH_INFO: `/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:45.646383 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b6c5f880-9203-461b-b32c-690dfa83e82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:45.647484 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb39a43-9c7d-4ed6-a7d2-cfdccce5ecb0 None admin] RBAC: Authorizing `identity:get_group(group_id=a7c4a8842bf64b77a6ed9d6b706b3c62)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:45.649175 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb39a43-9c7d-4ed6-a7d2-cfdccce5ecb0 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:45.651588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 219/439] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jul 20 08:59:45 2022] GET /identity/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:45.661298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb39a43-9c7d-4ed6-a7d2-cfdccce5ecb0 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:45.744915 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 221/440] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:45.768574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5f880-9203-461b-b32c-690dfa83e82e None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:45.798119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb39a43-9c7d-4ed6-a7d2-cfdccce5ecb0 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:45.830052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:45.838721 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:45.845494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf65aa-b993-43d9-8fd7-2fa9f9a8a2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:45.846395 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf65aa-b993-43d9-8fd7-2fa9f9a8a2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:45.846704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf65aa-b993-43d9-8fd7-2fa9f9a8a2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:45.847007 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf65aa-b993-43d9-8fd7-2fa9f9a8a2c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:45.847855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf65aa-b993-43d9-8fd7-2fa9f9a8a2c6 None admin] RBAC: Adding query filter params (group.id=a7c4a8842bf64b77a6ed9d6b706b3c62, role.id=05ca4843c34e463c94de09f756d95318, scope.project.id=81fa154d1fc04266abee9f0051cba2fa) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:45.848171 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf65aa-b993-43d9-8fd7-2fa9f9a8a2c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:45.849613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf65aa-b993-43d9-8fd7-2fa9f9a8a2c6 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:45.858223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 220/441] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jul 20 08:59:45 2022] GET /identity/v3/role_assignments?group.id=a7c4a8842bf64b77a6ed9d6b706b3c62&scope.project.id=81fa154d1fc04266abee9f0051cba2fa&role.id=05ca4843c34e463c94de09f756d95318 => generated 611 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:45.880722 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1682f-704c-47da-99ca-3b22ce126ce1 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:45.881375 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1682f-704c-47da-99ca-3b22ce126ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:45.881375 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1682f-704c-47da-99ca-3b22ce126ce1 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:45.883013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 221/442] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:59:45.888766 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab80da-6115-48e3-95c7-188a8adf23b5 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:45.889168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab80da-6115-48e3-95c7-188a8adf23b5 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:45.889458 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab80da-6115-48e3-95c7-188a8adf23b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:45.907823 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-5dab80da-6115-48e3-95c7-188a8adf23b5 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:45.909059 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5dab80da-6115-48e3-95c7-188a8adf23b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:45.918172 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5f880-9203-461b-b32c-690dfa83e82e None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:45.957333 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb1f98-ed44-45ee-bf91-0afc105ab133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:45.958157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb1f98-ed44-45ee-bf91-0afc105ab133 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:45.958440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb1f98-ed44-45ee-bf91-0afc105ab133 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:45.958741 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb1f98-ed44-45ee-bf91-0afc105ab133 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:45.971141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb1f98-ed44-45ee-bf91-0afc105ab133 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:45.979380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-e6cb1f98-ed44-45ee-bf91-0afc105ab133 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:45.979380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:45.979380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:45.979380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:45.979380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:45.979380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:45.979380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:45.979380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:45.979380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:45.979380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:45.979380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:45.979380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:45.979380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:45.979380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:45.979380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:45.979380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:45.980351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:45.980351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:45.980351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:45.980351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:45.980351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:45.980351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:45.980351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:45.980351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:45.980351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:45.980351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:45.980351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:45.980351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:45.980351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:45.980351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:45.980351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:45.980351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:45.981256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:45.981256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:45.981256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:45.981256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:45.981256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:45.981256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:45.981256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:45.981256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:45.981256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:45.981256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:45.981256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:45.981256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:45.981256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 222/443] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 08:59:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:45.987066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 222/444] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:59:45.994808 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb1f98-ed44-45ee-bf91-0afc105ab133 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:46.011275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab80da-6115-48e3-95c7-188a8adf23b5 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:46.112955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb1f98-ed44-45ee-bf91-0afc105ab133 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:46.115448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:46.143532 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab80da-6115-48e3-95c7-188a8adf23b5 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:46.152993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3991e8-2611-4e58-9210-0b3c24a38f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:46.153663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3991e8-2611-4e58-9210-0b3c24a38f41 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:46.153949 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3991e8-2611-4e58-9210-0b3c24a38f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:46.154226 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3991e8-2611-4e58-9210-0b3c24a38f41 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:46.155040 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3991e8-2611-4e58-9210-0b3c24a38f41 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:46.155347 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3991e8-2611-4e58-9210-0b3c24a38f41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:46.156727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3991e8-2611-4e58-9210-0b3c24a38f41 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:46.166762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 223/445] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:46.171373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:46.172666 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3991e8-2611-4e58-9210-0b3c24a38f41 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:46.188930 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e5254-1cfb-4422-a2d7-33abe8fc503e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:46.189732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e5254-1cfb-4422-a2d7-33abe8fc503e None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:46.190041 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e5254-1cfb-4422-a2d7-33abe8fc503e None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:46.190315 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e5254-1cfb-4422-a2d7-33abe8fc503e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:46.201119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e5254-1cfb-4422-a2d7-33abe8fc503e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:46.209476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-420e5254-1cfb-4422-a2d7-33abe8fc503e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:46.209476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:46.209476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:46.209476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:46.209476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:46.209476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:46.209476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:46.209476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:46.209476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:46.209476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:46.209476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:46.209476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:46.209476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:46.209476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:46.209476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:46.209476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:46.210420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:46.210420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:46.210420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:46.210420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:46.210420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:46.210420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:46.210420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:46.210420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:46.210420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:46.210420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:46.210420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:46.210420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:46.210420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:46.210420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:46.210420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:46.210420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:46.211300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:46.211300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:46.211300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:46.211300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:46.211300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:46.211300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:46.211300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:46.211300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:46.211300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:46.211300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:46.211300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:59:46.211300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:46.211300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 223/446] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 08:59:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:46.216354 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e5254-1cfb-4422-a2d7-33abe8fc503e None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:46.293702 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3991e8-2611-4e58-9210-0b3c24a38f41 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:46.332024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e398b05-097b-4524-8c6b-1dfea171b9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:46.332682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e398b05-097b-4524-8c6b-1dfea171b9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:46.332973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e398b05-097b-4524-8c6b-1dfea171b9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:46.333258 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e398b05-097b-4524-8c6b-1dfea171b9f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:46.344570 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e5254-1cfb-4422-a2d7-33abe8fc503e None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:46.374984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:46.386382 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150df17-778a-406e-8e84-99a806b17150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:46.387099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150df17-778a-406e-8e84-99a806b17150 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:46.387412 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150df17-778a-406e-8e84-99a806b17150 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:46.387777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150df17-778a-406e-8e84-99a806b17150 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:46.388540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150df17-778a-406e-8e84-99a806b17150 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:46.388850 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150df17-778a-406e-8e84-99a806b17150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:46.390620 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150df17-778a-406e-8e84-99a806b17150 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:46.400324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 224/447] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:46.406467 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150df17-778a-406e-8e84-99a806b17150 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:46.410229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e398b05-097b-4524-8c6b-1dfea171b9f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:46.411718 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e398b05-097b-4524-8c6b-1dfea171b9f4 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:46.453256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:46.507500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150df17-778a-406e-8e84-99a806b17150 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:46.508894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 224/448] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:46.514405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e398b05-097b-4524-8c6b-1dfea171b9f4 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:46.541997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516eda3b-798a-498a-b671-9e7f88cc2102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:46.542652 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516eda3b-798a-498a-b671-9e7f88cc2102 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:46.542969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516eda3b-798a-498a-b671-9e7f88cc2102 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:46.543252 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516eda3b-798a-498a-b671-9e7f88cc2102 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:46.579297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:46.613615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516eda3b-798a-498a-b671-9e7f88cc2102 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:46.615103 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516eda3b-798a-498a-b671-9e7f88cc2102 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:46.615247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e398b05-097b-4524-8c6b-1dfea171b9f4 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:46.648952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44051d-7517-44db-b266-4b0a99573481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:46.649803 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44051d-7517-44db-b266-4b0a99573481 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:46.650089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44051d-7517-44db-b266-4b0a99573481 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:46.650358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44051d-7517-44db-b266-4b0a99573481 None admin] PATH_INFO: `/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:46.651561 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44051d-7517-44db-b266-4b0a99573481 None admin] RBAC: Authorizing `identity:get_project(project_id=9845abb3fbe543cbbeaf3f3c167ea3a7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:46.652982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44051d-7517-44db-b266-4b0a99573481 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:46.659855 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 225/449] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:46 2022] GET /identity/v3/projects/9845abb3fbe543cbbeaf3f3c167ea3a7 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:46.660938 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44051d-7517-44db-b266-4b0a99573481 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:46.662774 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:46.718319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 225/450] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 08:59:46.723996 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516eda3b-798a-498a-b671-9e7f88cc2102 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:46.760294 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44051d-7517-44db-b266-4b0a99573481 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:46.793516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51399dfe-ea2e-49e7-8384-f45236875f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:46.794373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51399dfe-ea2e-49e7-8384-f45236875f5b None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:46.794658 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51399dfe-ea2e-49e7-8384-f45236875f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:46.794959 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51399dfe-ea2e-49e7-8384-f45236875f5b None admin] PATH_INFO: `/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:46.796707 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51399dfe-ea2e-49e7-8384-f45236875f5b None admin] RBAC: Authorizing `identity:get_group(group_id=a7c4a8842bf64b77a6ed9d6b706b3c62)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:46.798145 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51399dfe-ea2e-49e7-8384-f45236875f5b None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:46.799937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 226/451] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jul 20 08:59:46 2022] GET /identity/v3/groups/a7c4a8842bf64b77a6ed9d6b706b3c62 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:46.805473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51399dfe-ea2e-49e7-8384-f45236875f5b None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:46.808207 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:46.823747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516eda3b-798a-498a-b671-9e7f88cc2102 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:46.858210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a161eb9-f475-4451-abd0-259daea6921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:46.859159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a161eb9-f475-4451-abd0-259daea6921e None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:46.859462 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a161eb9-f475-4451-abd0-259daea6921e None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:46.859723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a161eb9-f475-4451-abd0-259daea6921e None admin] PATH_INFO: `/v3/projects/d5e32bdaa14a4d31baa44b436b02a242` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:46.860895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a161eb9-f475-4451-abd0-259daea6921e None admin] RBAC: Authorizing `identity:get_project(project_id=d5e32bdaa14a4d31baa44b436b02a242)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:46.862304 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a161eb9-f475-4451-abd0-259daea6921e None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:46.864173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 226/452] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 08:59:46 2022] GET /identity/v3/projects/d5e32bdaa14a4d31baa44b436b02a242 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:46.869268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a161eb9-f475-4451-abd0-259daea6921e None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:46.883903 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:46.905189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51399dfe-ea2e-49e7-8384-f45236875f5b None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:46.939409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbce09-0f3e-4865-811b-9a65b8a352fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:46.940056 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbce09-0f3e-4865-811b-9a65b8a352fd None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:46.940335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbce09-0f3e-4865-811b-9a65b8a352fd None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:46.940610 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbce09-0f3e-4865-811b-9a65b8a352fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:46.941430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbce09-0f3e-4865-811b-9a65b8a352fd None admin] RBAC: Adding query filter params (group.id=a7c4a8842bf64b77a6ed9d6b706b3c62, role.id=05ca4843c34e463c94de09f756d95318, scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:46.941741 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbce09-0f3e-4865-811b-9a65b8a352fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:46.943312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbce09-0f3e-4865-811b-9a65b8a352fd None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:46.949454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 227/453] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jul 20 08:59:46 2022] GET /identity/v3/role_assignments?group.id=a7c4a8842bf64b77a6ed9d6b706b3c62&scope.project.id=9845abb3fbe543cbbeaf3f3c167ea3a7&role.id=05ca4843c34e463c94de09f756d95318 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:46.970132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a161eb9-f475-4451-abd0-259daea6921e None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:47.008135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfb041-507d-41f1-840c-eff3f8e3a7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:47.009008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfb041-507d-41f1-840c-eff3f8e3a7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:47.009285 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfb041-507d-41f1-840c-eff3f8e3a7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:47.009559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfb041-507d-41f1-840c-eff3f8e3a7e8 None admin] PATH_INFO: `/v3/groups/3b88634c930c4fedbf0bab9055ee9ebe` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:47.010790 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfb041-507d-41f1-840c-eff3f8e3a7e8 None admin] RBAC: Authorizing `identity:get_group(group_id=3b88634c930c4fedbf0bab9055ee9ebe)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:47.012252 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfb041-507d-41f1-840c-eff3f8e3a7e8 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:47.013936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 227/454] 10.222.0.44 () {56 vars in 1180 bytes} [Wed Jul 20 08:59:46 2022] GET /identity/v3/groups/3b88634c930c4fedbf0bab9055ee9ebe => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:47.020626 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbce09-0f3e-4865-811b-9a65b8a352fd None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:47.090711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:47.128173 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbce09-0f3e-4865-811b-9a65b8a352fd None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:47.171603 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad449e67-41e8-4637-bc09-5a88b86dd89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:47.172264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad449e67-41e8-4637-bc09-5a88b86dd89a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:47.172545 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad449e67-41e8-4637-bc09-5a88b86dd89a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:47.172820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad449e67-41e8-4637-bc09-5a88b86dd89a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:47.173666 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad449e67-41e8-4637-bc09-5a88b86dd89a None admin] RBAC: Adding query filter params (group.id=3b88634c930c4fedbf0bab9055ee9ebe, role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=d5e32bdaa14a4d31baa44b436b02a242) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:47.173975 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad449e67-41e8-4637-bc09-5a88b86dd89a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:47.175455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad449e67-41e8-4637-bc09-5a88b86dd89a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:47.181346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 228/455] 10.222.0.44 () {56 vars in 1376 bytes} [Wed Jul 20 08:59:47 2022] GET /identity/v3/role_assignments?group.id=3b88634c930c4fedbf0bab9055ee9ebe&scope.project.id=d5e32bdaa14a4d31baa44b436b02a242&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:49.612784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8872d-596a-47a5-b21d-6578f18821af None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:49.613750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8872d-596a-47a5-b21d-6578f18821af None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:49.614238 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8872d-596a-47a5-b21d-6578f18821af None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:49.616136 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 228/456] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:49.621454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7d71e-325b-4988-bcd1-8f97a11b7379 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:49.621886 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7d71e-325b-4988-bcd1-8f97a11b7379 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:49.622192 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7d71e-325b-4988-bcd1-8f97a11b7379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:49.647393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-d6a7d71e-325b-4988-bcd1-8f97a11b7379 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:49.648616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d6a7d71e-325b-4988-bcd1-8f97a11b7379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:49.745429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 229/457] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:49.768361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e01110-31de-44cd-bb38-2db1037cd03e None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:49.768995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e01110-31de-44cd-bb38-2db1037cd03e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:49.769481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e01110-31de-44cd-bb38-2db1037cd03e None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:49.771363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 229/458] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:49.776809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc787450-9621-4417-9897-fcbb632f0d51 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:49.777142 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc787450-9621-4417-9897-fcbb632f0d51 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:49.777384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc787450-9621-4417-9897-fcbb632f0d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:49.782206 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e01110-31de-44cd-bb38-2db1037cd03e None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:49.808294 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-bc787450-9621-4417-9897-fcbb632f0d51 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:49.812032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-bc787450-9621-4417-9897-fcbb632f0d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:49.923054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:49.943399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:49.946456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 230/459] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:49.971195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e01110-31de-44cd-bb38-2db1037cd03e None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:49.994663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc787450-9621-4417-9897-fcbb632f0d51 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:50.033887 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855acbf9-a174-4d75-8c42-17cc5c832a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:50.035008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855acbf9-a174-4d75-8c42-17cc5c832a1b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:50.035442 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855acbf9-a174-4d75-8c42-17cc5c832a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:50.035866 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855acbf9-a174-4d75-8c42-17cc5c832a1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:50.047679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855acbf9-a174-4d75-8c42-17cc5c832a1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:50.050541 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855acbf9-a174-4d75-8c42-17cc5c832a1b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:50.084634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-855acbf9-a174-4d75-8c42-17cc5c832a1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:50.084634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:50.084634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:50.084634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:50.084634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:50.084634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:50.084634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:50.084634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:50.084634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:50.084634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:50.084634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:50.084634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:50.084634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:59:50.084634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:59:50.084634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:50.085623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:50.086471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:59:50.086471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:59:50.086471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:50.086471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:50.088553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 230/460] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:50.095720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855acbf9-a174-4d75-8c42-17cc5c832a1b None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:50.191041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc787450-9621-4417-9897-fcbb632f0d51 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:50.237058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17266649-acf5-4cb0-947b-1bbad849334b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:50.238795 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17266649-acf5-4cb0-947b-1bbad849334b None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:50.239264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17266649-acf5-4cb0-947b-1bbad849334b None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:50.239692 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17266649-acf5-4cb0-947b-1bbad849334b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:50.240339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855acbf9-a174-4d75-8c42-17cc5c832a1b None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:50.254203 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17266649-acf5-4cb0-947b-1bbad849334b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:50.256269 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17266649-acf5-4cb0-947b-1bbad849334b None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:50.276212 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:50.276385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-17266649-acf5-4cb0-947b-1bbad849334b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:50.276385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:50.276385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:50.276385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:50.276385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:50.276385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:50.276385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:50.276385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:50.276385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:50.276385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:50.276385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:50.276385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:50.276385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:59:50.276385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:59:50.276385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:50.277208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:50.278048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:59:50.278048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:59:50.278048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:50.278048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:50.279342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 231/461] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:50.280305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481e8e0-bf67-4296-a3ef-4d0918df5511 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:50.280566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481e8e0-bf67-4296-a3ef-4d0918df5511 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:50.280755 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481e8e0-bf67-4296-a3ef-4d0918df5511 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:50.282671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 232/462] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:50.283732 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0ad29-8950-4231-b7ed-8d21d7d97ec1 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:50.284016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0ad29-8950-4231-b7ed-8d21d7d97ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:50.284292 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0ad29-8950-4231-b7ed-8d21d7d97ec1 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:50.285618 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 233/463] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:50.286191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0ad29-8950-4231-b7ed-8d21d7d97ec1 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:50.291003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a05280-af21-4198-b100-0b3e5d1e1549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:50.291672 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a05280-af21-4198-b100-0b3e5d1e1549 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:50.291959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a05280-af21-4198-b100-0b3e5d1e1549 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:50.292239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a05280-af21-4198-b100-0b3e5d1e1549 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:50.292971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a05280-af21-4198-b100-0b3e5d1e1549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:50.293282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a05280-af21-4198-b100-0b3e5d1e1549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:50.294669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a05280-af21-4198-b100-0b3e5d1e1549 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:50.322870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 231/464] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 08:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:50.324910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb112c-dd8a-47f4-8197-f718ffb0f5d2 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:50.325389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb112c-dd8a-47f4-8197-f718ffb0f5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:50.325677 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb112c-dd8a-47f4-8197-f718ffb0f5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:50.343822 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-c9eb112c-dd8a-47f4-8197-f718ffb0f5d2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:50.344947 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c9eb112c-dd8a-47f4-8197-f718ffb0f5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:50.346150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:50.408750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0ad29-8950-4231-b7ed-8d21d7d97ec1 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:50.415806 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 232/465] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:50.423234 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b327f-6f01-45f5-88c3-fad2df138d19 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:50.423234 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b327f-6f01-45f5-88c3-fad2df138d19 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:50.423234 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b327f-6f01-45f5-88c3-fad2df138d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:50.438013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-d30b327f-6f01-45f5-88c3-fad2df138d19 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:50.440250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d30b327f-6f01-45f5-88c3-fad2df138d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:50.454972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6addf3-bfcf-4786-b0cc-391254f2b7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:50.455640 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6addf3-bfcf-4786-b0cc-391254f2b7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:50.455938 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6addf3-bfcf-4786-b0cc-391254f2b7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:50.456208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6addf3-bfcf-4786-b0cc-391254f2b7b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:50.456932 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6addf3-bfcf-4786-b0cc-391254f2b7b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:50.457243 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6addf3-bfcf-4786-b0cc-391254f2b7b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:50.458931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6addf3-bfcf-4786-b0cc-391254f2b7b3 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:50.475961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 234/466] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 08:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:50.475961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6addf3-bfcf-4786-b0cc-391254f2b7b3 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:50.523107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 233/467] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:50.523107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b327f-6f01-45f5-88c3-fad2df138d19 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:50.581320 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:50.599779 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6addf3-bfcf-4786-b0cc-391254f2b7b3 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:50.601342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:50.634208 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b327f-6f01-45f5-88c3-fad2df138d19 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:50.636232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcb9e7-3cd7-4263-a0a7-af6a01b13a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:50.637017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcb9e7-3cd7-4263-a0a7-af6a01b13a40 None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:50.637304 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcb9e7-3cd7-4263-a0a7-af6a01b13a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:50.637584 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcb9e7-3cd7-4263-a0a7-af6a01b13a40 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:50.638289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fcb9e7-3cd7-4263-a0a7-af6a01b13a40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:50.640538 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fcb9e7-3cd7-4263-a0a7-af6a01b13a40 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:50.662135 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 235/468] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 08:59:50 2022] POST /identity/v3/users => generated 246 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:59:50.666885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcb9e7-3cd7-4263-a0a7-af6a01b13a40 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:50.689357 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7e69f-7314-450a-bf56-b7612239886c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:50.690298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7e69f-7314-450a-bf56-b7612239886c None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:50.690586 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7e69f-7314-450a-bf56-b7612239886c None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:50.690889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7e69f-7314-450a-bf56-b7612239886c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:50.703474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f7e69f-7314-450a-bf56-b7612239886c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:50.705019 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f7e69f-7314-450a-bf56-b7612239886c None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:50.729556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-51f7e69f-7314-450a-bf56-b7612239886c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:50.729556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:50.729556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:50.729556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:50.729556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:50.729556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:50.729556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:50.729556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:50.729556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:50.729556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:50.729556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:50.729556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:50.729556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:59:50.729556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:59:50.729556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:50.730494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:50.731886 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:59:50.731886 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:59:50.731886 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:50.731886 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:50.737014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 234/469] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:50.738012 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7e69f-7314-450a-bf56-b7612239886c None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:50.820956 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcb9e7-3cd7-4263-a0a7-af6a01b13a40 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:50.866617 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af72448-640c-44d1-871c-f64a28176e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:50.867732 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af72448-640c-44d1-871c-f64a28176e9a None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:50.868433 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af72448-640c-44d1-871c-f64a28176e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:50.869436 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af72448-640c-44d1-871c-f64a28176e9a None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:50.870568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af72448-640c-44d1-871c-f64a28176e9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:50.871855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7e69f-7314-450a-bf56-b7612239886c None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:50.872359 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af72448-640c-44d1-871c-f64a28176e9a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:50.875269 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:50.901264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 236/470] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 08:59:50 2022] POST /identity/v3/users => generated 248 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:59:50.902102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af72448-640c-44d1-871c-f64a28176e9a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:50.921852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:50.924071 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7501ca-5ee2-47bf-aa15-8bbbf01fba43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:50.925121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7501ca-5ee2-47bf-aa15-8bbbf01fba43 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:50.925555 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7501ca-5ee2-47bf-aa15-8bbbf01fba43 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:50.925972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7501ca-5ee2-47bf-aa15-8bbbf01fba43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:50.938392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7501ca-5ee2-47bf-aa15-8bbbf01fba43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:50.940325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7501ca-5ee2-47bf-aa15-8bbbf01fba43 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:50.964147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-6e7501ca-5ee2-47bf-aa15-8bbbf01fba43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:50.964147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:50.964147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:50.964147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:50.964147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:50.964147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:50.964147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:50.964147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:50.964147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:50.964147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:50.964147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:50.964147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:50.964147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:59:50.964147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:59:50.964147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:50.965253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:50.966277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:59:50.966277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:59:50.966277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:50.966277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:50.968828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 235/471] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:50.977463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7501ca-5ee2-47bf-aa15-8bbbf01fba43 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:51.041283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af72448-640c-44d1-871c-f64a28176e9a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:51.094971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7501ca-5ee2-47bf-aa15-8bbbf01fba43 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:51.096115 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1324bbb-1c85-4167-a887-54242a50f279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:51.096804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324bbb-1c85-4167-a887-54242a50f279 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:51.097087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324bbb-1c85-4167-a887-54242a50f279 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:51.097366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1324bbb-1c85-4167-a887-54242a50f279 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:51.098108 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1324bbb-1c85-4167-a887-54242a50f279 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:51.098420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1324bbb-1c85-4167-a887-54242a50f279 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:51.099848 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1324bbb-1c85-4167-a887-54242a50f279 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:51.113900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 237/472] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 08:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:51.119201 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1324bbb-1c85-4167-a887-54242a50f279 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:51.133611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e7b30-b0eb-4868-a7bf-74a7c15eeb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:51.134285 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e7b30-b0eb-4868-a7bf-74a7c15eeb0a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:51.134571 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e7b30-b0eb-4868-a7bf-74a7c15eeb0a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:51.134892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e7b30-b0eb-4868-a7bf-74a7c15eeb0a None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:51.135692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e7b30-b0eb-4868-a7bf-74a7c15eeb0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:51.136007 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e7b30-b0eb-4868-a7bf-74a7c15eeb0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:51.137445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e7b30-b0eb-4868-a7bf-74a7c15eeb0a None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:51.150556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 236/473] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 08:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:51.157803 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e7b30-b0eb-4868-a7bf-74a7c15eeb0a None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:51.182162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:51.222747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:51.255896 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1324bbb-1c85-4167-a887-54242a50f279 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:51.297324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e7b30-b0eb-4868-a7bf-74a7c15eeb0a None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:51.305032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85b535-1583-471c-9672-2cfaeaa14214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:51.305728 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85b535-1583-471c-9672-2cfaeaa14214 None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:51.306018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85b535-1583-471c-9672-2cfaeaa14214 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:51.306301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85b535-1583-471c-9672-2cfaeaa14214 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:51.307058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85b535-1583-471c-9672-2cfaeaa14214 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:51.309189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85b535-1583-471c-9672-2cfaeaa14214 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:51.336897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e626e-3dd4-4828-96bb-2c0edaa05401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:51.337625 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e626e-3dd4-4828-96bb-2c0edaa05401 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:51.337914 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e626e-3dd4-4828-96bb-2c0edaa05401 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:51.338196 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e626e-3dd4-4828-96bb-2c0edaa05401 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:51.339164 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e626e-3dd4-4828-96bb-2c0edaa05401 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:51.339270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 238/474] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 08:59:51 2022] POST /identity/v3/users => generated 249 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:59:51.340389 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e58e4-22a3-4a41-ba1b-517e474e8d76 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:51.340649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e626e-3dd4-4828-96bb-2c0edaa05401 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:51.340752 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e58e4-22a3-4a41-ba1b-517e474e8d76 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:51.340926 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e58e4-22a3-4a41-ba1b-517e474e8d76 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:51.343327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 239/475] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:51.352307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece5492-9487-4bc4-b55b-13d5c4787284 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:51.352630 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece5492-9487-4bc4-b55b-13d5c4787284 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:51.352877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece5492-9487-4bc4-b55b-13d5c4787284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:51.378969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 237/476] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 08:59:51 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:59:51.393965 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-6ece5492-9487-4bc4-b55b-13d5c4787284 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:51.395231 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6ece5492-9487-4bc4-b55b-13d5c4787284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:51.479143 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 240/477] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:51.515553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e626e-3dd4-4828-96bb-2c0edaa05401 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:51.597248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:51.633085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e626e-3dd4-4828-96bb-2c0edaa05401 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:51.675589 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28238302-86ce-45b9-a19b-a6385b8b2cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:51.676367 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28238302-86ce-45b9-a19b-a6385b8b2cd2 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:51.676542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28238302-86ce-45b9-a19b-a6385b8b2cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:51.676815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28238302-86ce-45b9-a19b-a6385b8b2cd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:51.677560 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28238302-86ce-45b9-a19b-a6385b8b2cd2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:51.679005 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28238302-86ce-45b9-a19b-a6385b8b2cd2 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:51.688725 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.sql.core [None req-28238302-86ce-45b9-a19b-a6385b8b2cd2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 08:59:51.688725 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 08:59:51.688725 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [parameters: {'id': '383bcfebdeb741a5b8720d94f59db1b7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 08:59:51.688725 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80334) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 08:59:51.706287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-28238302-86ce-45b9-a19b-a6385b8b2cd2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 08:59:51.706287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:51.706287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 08:59:51.706287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 08:59:51.706287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 08:59:51.706287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 08:59:51.706287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 08:59:51.706287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 08:59:51.706287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 08:59:51.706287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application conn.query(q) Jul 20 08:59:51.706287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 08:59:51.706287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 08:59:51.706287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 08:59:51.706287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application result.read() Jul 20 08:59:51.706287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return ref.to_dict() Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 08:59:51.707265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application next(self.gen) Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application yield resource Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application next(self.gen) Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise self.value Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application session.commit() Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 08:59:51.708418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self.session.flush() Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application compat.raise_( Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 08:59:51.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 08:59:51.710902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 08:59:51.710902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 08:59:51.710902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 08:59:51.710902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 08:59:51.710902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 08:59:51.710902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 08:59:51.710902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 08:59:51.710902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 08:59:51.710902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 08:59:51.710902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 08:59:51.710902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 08:59:51.710902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 08:59:51.710902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 08:59:51.710902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 08:59:51.710902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 08:59:51.710902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception Jul 20 08:59:51.711709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 08:59:51.711709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 08:59:51.711709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 08:59:51.711709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 08:59:51.711709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 08:59:51.711709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 08:59:51.711709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 08:59:51.711709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application conn.query(q) Jul 20 08:59:51.711709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 08:59:51.711709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 08:59:51.711709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 08:59:51.711709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application result.read() Jul 20 08:59:51.711709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 08:59:51.711709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 08:59:51.711709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 08:59:51.711709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 08:59:51.713182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 08:59:51.713182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 08:59:51.713182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 08:59:51.713182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 08:59:51.713182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 08:59:51.713182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 08:59:51.713182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [parameters: {'id': '383bcfebdeb741a5b8720d94f59db1b7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 08:59:51.713182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 08:59:51.713182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:51.713182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:59:51.713182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:51.713182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:51.713182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:51.713182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:51.713182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:51.713182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:51.714217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:51.714217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:51.714217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:51.714217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:51.714217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:51.714217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:51.714217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 20 08:59:51.714217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 20 08:59:51.714217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:51.714217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:51.714217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 20 08:59:51.714217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 20 08:59:51.714217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 08:59:51.714217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 08:59:51.714217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 08:59:51.714217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:51.715322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 238/478] 10.222.0.44 () {60 vars in 1131 bytes} [Wed Jul 20 08:59:51 2022] POST /identity/v3/roles => generated 144 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:51.718723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece5492-9487-4bc4-b55b-13d5c4787284 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:51.788015 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:51.836443 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece5492-9487-4bc4-b55b-13d5c4787284 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:51.878399 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3adf6-d2ad-4dcd-9c98-89ae63dd82a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:51.879284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3adf6-d2ad-4dcd-9c98-89ae63dd82a3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:51.879564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3adf6-d2ad-4dcd-9c98-89ae63dd82a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:51.879886 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3adf6-d2ad-4dcd-9c98-89ae63dd82a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:51.905889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3adf6-d2ad-4dcd-9c98-89ae63dd82a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:51.921535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-2fa3adf6-d2ad-4dcd-9c98-89ae63dd82a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:51.921535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:51.921535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:51.921535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:51.921535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:51.921535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:51.921535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:51.921535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:51.921535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:51.921535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:51.921535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:51.921535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:51.921535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:51.921535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:51.921535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:51.921535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:51.922425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:51.922425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:51.922425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:51.922425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:51.922425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:51.922425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:51.922425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:51.922425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:51.922425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:51.922425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:51.922425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:51.922425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:51.922425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:51.922425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:51.922425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:51.922425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:51.923247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:51.923247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:51.923247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:51.923247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:51.923247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:51.923247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:51.923247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:51.923247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:51.923247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:51.923247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:51.923247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:59:51.923247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:51.924318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 241/479] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 08:59:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:51.933635 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28238302-86ce-45b9-a19b-a6385b8b2cd2 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:52.132384 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28238302-86ce-45b9-a19b-a6385b8b2cd2 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:52.201168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777acf1d-4829-4b98-b999-b67cbced2a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:52.202798 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777acf1d-4829-4b98-b999-b67cbced2a07 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:52.203294 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777acf1d-4829-4b98-b999-b67cbced2a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:52.203784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777acf1d-4829-4b98-b999-b67cbced2a07 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:52.205177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777acf1d-4829-4b98-b999-b67cbced2a07 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:52.205721 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777acf1d-4829-4b98-b999-b67cbced2a07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:52.212320 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777acf1d-4829-4b98-b999-b67cbced2a07 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:52.216493 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:52.232900 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 239/480] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:53.424508 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b357a-3c0b-4108-beca-354d0b7290af None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:53.425025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b357a-3c0b-4108-beca-354d0b7290af None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:53.425128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b357a-3c0b-4108-beca-354d0b7290af None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:53.426703 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 242/481] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:53.432867 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebff74-cd20-46ba-b7f4-e2eb0693f531 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:53.433363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebff74-cd20-46ba-b7f4-e2eb0693f531 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:53.433938 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebff74-cd20-46ba-b7f4-e2eb0693f531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:53.466505 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-72ebff74-cd20-46ba-b7f4-e2eb0693f531 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:53.469102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-72ebff74-cd20-46ba-b7f4-e2eb0693f531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:53.586359 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 240/482] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:53.618457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b357a-3c0b-4108-beca-354d0b7290af None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:53.823808 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b357a-3c0b-4108-beca-354d0b7290af None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:53.851090 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:53.872392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06219dc-14e2-4feb-a275-8e0d3f6dba9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:53.873612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06219dc-14e2-4feb-a275-8e0d3f6dba9a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:53.874101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06219dc-14e2-4feb-a275-8e0d3f6dba9a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:53.874605 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06219dc-14e2-4feb-a275-8e0d3f6dba9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:53.891546 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06219dc-14e2-4feb-a275-8e0d3f6dba9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:53.899736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-c06219dc-14e2-4feb-a275-8e0d3f6dba9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:59:53.899736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:53.899736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:53.899736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:53.899736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:53.899736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:53.899736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:53.899736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:53.899736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:53.899736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:53.899736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:53.899736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:53.899736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:53.899736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:53.899736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:53.899736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:53.900620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:53.900620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:53.900620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:53.900620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:53.900620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:53.900620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:53.900620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:53.900620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:53.900620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:53.900620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:53.900620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:53.900620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:53.900620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:53.900620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:53.900620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:53.900620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:53.901412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:53.901412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:53.901412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:53.901412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:53.901412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:53.901412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:53.901412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:53.901412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:53.901412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:53.901412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:53.901412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:59:53.901412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:53.903769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 243/483] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 08:59:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:53.910286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebff74-cd20-46ba-b7f4-e2eb0693f531 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:54.127549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebff74-cd20-46ba-b7f4-e2eb0693f531 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:54.170931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2efaf0-404b-46e4-82f1-831ef9d926f6 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:54.171588 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2efaf0-404b-46e4-82f1-831ef9d926f6 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:54.172075 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2efaf0-404b-46e4-82f1-831ef9d926f6 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:54.173902 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 244/484] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:59:54.179308 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5b23a-f0fa-43ea-8e35-dc36a5546605 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:54.179598 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5b23a-f0fa-43ea-8e35-dc36a5546605 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:54.179879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5b23a-f0fa-43ea-8e35-dc36a5546605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:54.199143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:54.224817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d2f3f-0dab-4552-8867-04407fb46231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:54.225903 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d2f3f-0dab-4552-8867-04407fb46231 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:54.226388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d2f3f-0dab-4552-8867-04407fb46231 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:54.226910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d2f3f-0dab-4552-8867-04407fb46231 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:54.228235 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d2f3f-0dab-4552-8867-04407fb46231 None admin] RBAC: Adding query filter params (name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:54.228752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d2f3f-0dab-4552-8867-04407fb46231 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:54.231240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d2f3f-0dab-4552-8867-04407fb46231 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:54.232126 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-9fc5b23a-f0fa-43ea-8e35-dc36a5546605 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:54.234800 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9fc5b23a-f0fa-43ea-8e35-dc36a5546605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:54.247551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 241/485] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 08:59:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:54.253193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d2f3f-0dab-4552-8867-04407fb46231 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:54.375781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 245/486] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:54.403495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5b23a-f0fa-43ea-8e35-dc36a5546605 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:54.447886 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d2f3f-0dab-4552-8867-04407fb46231 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:54.518064 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d3cb5-4089-4e30-9b8f-5f0149a89540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:54.518903 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d3cb5-4089-4e30-9b8f-5f0149a89540 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:54.519463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d3cb5-4089-4e30-9b8f-5f0149a89540 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:54.519746 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d3cb5-4089-4e30-9b8f-5f0149a89540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:54.533781 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d3cb5-4089-4e30-9b8f-5f0149a89540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:54.541739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d3cb5-4089-4e30-9b8f-5f0149a89540 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:54.562023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5b23a-f0fa-43ea-8e35-dc36a5546605 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:54.568015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-1a8d3cb5-4089-4e30-9b8f-5f0149a89540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:54.568015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:54.568015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:54.568015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:54.568015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:54.568015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:54.568015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:54.568015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:54.568015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:54.568015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:54.568015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:54.568015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:54.568015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:59:54.568015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:59:54.568015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:54.568971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:54.570240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:59:54.570240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:59:54.570240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:54.570240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:54.570240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 242/487] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:54.571210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a6a12-0717-4571-9d9e-13a951fdab1b None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:54.571596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a6a12-0717-4571-9d9e-13a951fdab1b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:54.571874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a6a12-0717-4571-9d9e-13a951fdab1b None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:54.574748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 243/488] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:54.575676 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a6a12-0717-4571-9d9e-13a951fdab1b None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:54.583078 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:54.622700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c8d6a-7742-43cb-836b-54029339198b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:54.624136 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c8d6a-7742-43cb-836b-54029339198b None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:54.624663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c8d6a-7742-43cb-836b-54029339198b None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:54.625210 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c8d6a-7742-43cb-836b-54029339198b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:54.636650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:54.641748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c8d6a-7742-43cb-836b-54029339198b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:54.650284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-e20c8d6a-7742-43cb-836b-54029339198b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:59:54.650284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:54.650284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:54.650284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:54.650284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:54.650284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:54.650284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:54.650284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:54.650284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:54.650284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:54.650284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:54.650284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:54.650284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:54.650284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:54.650284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:54.650284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:54.651372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:54.651372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:54.651372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:54.651372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:54.651372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:54.651372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:54.651372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:54.651372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:54.651372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:54.651372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:54.651372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:54.651372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:54.651372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:54.651372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:54.651372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:54.651372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:54.652341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:54.652341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:54.652341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:54.652341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:54.652341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:54.652341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:54.652341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:54.652341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:54.652341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:54.652341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:54.652341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:59:54.652341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:54.652341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 246/489] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 08:59:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:59:54.654392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce9fc8-072d-41f0-b59b-b1a37c3e3da6 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:54.655320 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce9fc8-072d-41f0-b59b-b1a37c3e3da6 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:54.655710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce9fc8-072d-41f0-b59b-b1a37c3e3da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:54.673440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-a6ce9fc8-072d-41f0-b59b-b1a37c3e3da6 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:54.674656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-a6ce9fc8-072d-41f0-b59b-b1a37c3e3da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:54.721189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a6a12-0717-4571-9d9e-13a951fdab1b None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:54.749236 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 247/490] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:54.755317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce9fc8-072d-41f0-b59b-b1a37c3e3da6 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:54.765453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2288da5-0f21-4d6d-b766-a8fac37dabf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:54.766360 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2288da5-0f21-4d6d-b766-a8fac37dabf9 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:54.766816 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2288da5-0f21-4d6d-b766-a8fac37dabf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:54.767238 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2288da5-0f21-4d6d-b766-a8fac37dabf9 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:54.768114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2288da5-0f21-4d6d-b766-a8fac37dabf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:54.769311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2288da5-0f21-4d6d-b766-a8fac37dabf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:54.770883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2288da5-0f21-4d6d-b766-a8fac37dabf9 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:54.785922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 244/491] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 08:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:54.791555 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2288da5-0f21-4d6d-b766-a8fac37dabf9 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:54.879020 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:54.882141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce9fc8-072d-41f0-b59b-b1a37c3e3da6 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:54.901688 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2288da5-0f21-4d6d-b766-a8fac37dabf9 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:54.903240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:54.918153 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d6fd0-c80f-437c-b0b6-da31a2fb8641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:54.918782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d6fd0-c80f-437c-b0b6-da31a2fb8641 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:54.919067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d6fd0-c80f-437c-b0b6-da31a2fb8641 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:54.919343 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d6fd0-c80f-437c-b0b6-da31a2fb8641 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:54.920672 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d6fd0-c80f-437c-b0b6-da31a2fb8641 None admin] RBAC: Adding query filter params (name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:54.921079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d6fd0-c80f-437c-b0b6-da31a2fb8641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:54.922458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d6fd0-c80f-437c-b0b6-da31a2fb8641 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:54.934749 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 248/492] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 08:59:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:54.934749 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d6fd0-c80f-437c-b0b6-da31a2fb8641 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:54.944512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96008a46-8985-449b-bbb1-1f1e926cdca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:54.945411 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96008a46-8985-449b-bbb1-1f1e926cdca0 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:54.945702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96008a46-8985-449b-bbb1-1f1e926cdca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:54.945981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96008a46-8985-449b-bbb1-1f1e926cdca0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:54.953701 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96008a46-8985-449b-bbb1-1f1e926cdca0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:54.955311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96008a46-8985-449b-bbb1-1f1e926cdca0 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:54.972680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-96008a46-8985-449b-bbb1-1f1e926cdca0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:59:54.972680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:54.972680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:54.972680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:54.972680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:54.972680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:54.972680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:54.972680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:54.972680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:54.972680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:54.972680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:54.972680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:54.972680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:59:54.972680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:59:54.972680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:59:54.972680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:54.973602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:54.974481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:54.974481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:54.974481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:59:54.974481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:59:54.974481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:59:54.974481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:59:54.974481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:59:54.974481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:59:54.974481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:59:54.974481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:54.974481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 245/493] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jul 20 08:59:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:54.975594 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80229a33-184b-4fd5-9791-8d8bbf63e22e None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:54.975803 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80229a33-184b-4fd5-9791-8d8bbf63e22e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:54.976020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80229a33-184b-4fd5-9791-8d8bbf63e22e None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:54.982217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 246/494] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:54.982217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80229a33-184b-4fd5-9791-8d8bbf63e22e None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:55.048295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d6fd0-c80f-437c-b0b6-da31a2fb8641 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:55.083493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d87a7e-87da-43d5-9140-57b4c84a30ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:55.084287 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d87a7e-87da-43d5-9140-57b4c84a30ec None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:55.084572 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d87a7e-87da-43d5-9140-57b4c84a30ec None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:55.084854 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d87a7e-87da-43d5-9140-57b4c84a30ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:55.088972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80229a33-184b-4fd5-9791-8d8bbf63e22e None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:55.096103 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d87a7e-87da-43d5-9140-57b4c84a30ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:55.103830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-10d87a7e-87da-43d5-9140-57b4c84a30ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:59:55.103830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:55.103830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:55.103830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:55.103830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:55.103830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:55.103830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:55.103830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:55.103830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:55.103830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:55.103830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:55.103830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:55.103830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:55.103830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:55.103830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:55.103830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:55.104981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:55.104981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:55.104981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:55.104981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:55.104981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:55.104981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:55.104981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:55.104981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:55.104981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:55.104981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:55.104981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:55.104981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:55.104981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:55.104981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:55.104981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:55.104981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:55.105804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:55.105804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:55.105804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:55.105804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:55.105804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:55.105804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:55.105804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:55.105804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:55.105804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:55.105804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:55.105804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:59:55.105804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:55.105804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 249/495] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 08:59:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:55.105804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d87a7e-87da-43d5-9140-57b4c84a30ec None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:55.132331 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa83e8-48e9-4dcb-b78a-a28bce5ae8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:55.133119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa83e8-48e9-4dcb-b78a-a28bce5ae8ad None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:55.133403 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa83e8-48e9-4dcb-b78a-a28bce5ae8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:55.133681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa83e8-48e9-4dcb-b78a-a28bce5ae8ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:55.134490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:55.144451 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa83e8-48e9-4dcb-b78a-a28bce5ae8ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:55.145922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa83e8-48e9-4dcb-b78a-a28bce5ae8ad None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:55.163763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-c6fa83e8-48e9-4dcb-b78a-a28bce5ae8ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:55.163763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:55.163763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:55.163763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:55.163763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:55.163763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:55.163763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:55.163763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:55.163763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:55.163763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:55.163763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:55.163763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:55.163763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:59:55.163763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:59:55.163763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:55.164779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:55.165649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:59:55.165649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:59:55.165649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:55.165649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:55.165649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 247/496] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:55.166191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd6d15-7eae-4c54-9fcb-0d3c79aea971 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:55.166483 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd6d15-7eae-4c54-9fcb-0d3c79aea971 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:55.166852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd6d15-7eae-4c54-9fcb-0d3c79aea971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:55.183580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-efcd6d15-7eae-4c54-9fcb-0d3c79aea971 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:55.184696 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-efcd6d15-7eae-4c54-9fcb-0d3c79aea971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:55.194905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:55.215036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d87a7e-87da-43d5-9140-57b4c84a30ec None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:55.247570 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948ede7-755c-4cfc-841d-033f5ad0dc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:55.248791 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948ede7-755c-4cfc-841d-033f5ad0dc13 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:55.249081 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948ede7-755c-4cfc-841d-033f5ad0dc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:55.249362 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948ede7-755c-4cfc-841d-033f5ad0dc13 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:55.250201 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948ede7-755c-4cfc-841d-033f5ad0dc13 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:55.250511 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948ede7-755c-4cfc-841d-033f5ad0dc13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:55.251962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948ede7-755c-4cfc-841d-033f5ad0dc13 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:55.254098 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 248/497] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:55.254969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd6d15-7eae-4c54-9fcb-0d3c79aea971 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:55.292623 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 250/498] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jul 20 08:59:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:55.294007 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26a8fa-be0a-44c1-ab65-cdf35f79062e None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:55.294298 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26a8fa-be0a-44c1-ab65-cdf35f79062e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:55.294578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26a8fa-be0a-44c1-ab65-cdf35f79062e None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:55.298348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 251/499] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:55.298348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26a8fa-be0a-44c1-ab65-cdf35f79062e None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:55.355675 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd6d15-7eae-4c54-9fcb-0d3c79aea971 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:55.380621 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:55.391481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbffe7e-954d-4ecf-8fca-83055fcfb5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:55.392146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbffe7e-954d-4ecf-8fca-83055fcfb5db None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:55.392427 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbffe7e-954d-4ecf-8fca-83055fcfb5db None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:55.392707 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbffe7e-954d-4ecf-8fca-83055fcfb5db None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:55.393452 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbffe7e-954d-4ecf-8fca-83055fcfb5db None admin] RBAC: Adding query filter params (name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:55.393786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbffe7e-954d-4ecf-8fca-83055fcfb5db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:55.395214 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbffe7e-954d-4ecf-8fca-83055fcfb5db None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:55.400935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26a8fa-be0a-44c1-ab65-cdf35f79062e None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:55.403101 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 249/500] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 08:59:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:55.403953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbffe7e-954d-4ecf-8fca-83055fcfb5db None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:55.414247 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:55.435495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58ebee-2f69-4bb1-ad25-ef0a441fce75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:55.436616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58ebee-2f69-4bb1-ad25-ef0a441fce75 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:55.436893 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58ebee-2f69-4bb1-ad25-ef0a441fce75 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:55.437174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58ebee-2f69-4bb1-ad25-ef0a441fce75 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:55.437892 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe58ebee-2f69-4bb1-ad25-ef0a441fce75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:55.438199 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe58ebee-2f69-4bb1-ad25-ef0a441fce75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:55.439611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe58ebee-2f69-4bb1-ad25-ef0a441fce75 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:55.450912 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 252/501] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 08:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:55.452273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58ebee-2f69-4bb1-ad25-ef0a441fce75 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:55.511640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbffe7e-954d-4ecf-8fca-83055fcfb5db None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:55.549670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58ebee-2f69-4bb1-ad25-ef0a441fce75 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:55.551382 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aec3a8-b04b-458a-8b39-42cf5022fefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:55.552183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aec3a8-b04b-458a-8b39-42cf5022fefa None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:55.552463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aec3a8-b04b-458a-8b39-42cf5022fefa None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:55.552732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aec3a8-b04b-458a-8b39-42cf5022fefa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:55.563406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aec3a8-b04b-458a-8b39-42cf5022fefa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:55.571299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-45aec3a8-b04b-458a-8b39-42cf5022fefa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:59:55.571299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:55.571299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:55.571299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:55.571299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:55.571299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:55.571299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:55.571299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:55.571299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:55.571299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:55.571299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:55.571299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:55.571299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:55.571299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:55.571299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:55.571299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:55.572356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:55.572356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:55.572356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:55.572356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:55.572356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:55.572356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:55.572356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:55.572356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:55.572356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:55.572356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:55.572356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:55.572356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:55.572356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:55.572356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:55.572356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:55.572356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:55.573180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:55.573180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:55.573180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:55.573180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:55.573180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:55.573180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:55.573180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:55.573180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:55.573180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:55.573180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:55.573180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:59:55.573180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:55.573180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 250/502] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 08:59:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:55.574018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20e1df-9eea-4ae9-aaba-462e5b4202c7 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:55.574018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20e1df-9eea-4ae9-aaba-462e5b4202c7 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:55.574149 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20e1df-9eea-4ae9-aaba-462e5b4202c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:55.589223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9d276-a3fe-4787-988e-1a818bdcfba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:55.589876 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9d276-a3fe-4787-988e-1a818bdcfba0 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:55.590156 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9d276-a3fe-4787-988e-1a818bdcfba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:55.590439 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9d276-a3fe-4787-988e-1a818bdcfba0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:55.590725 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-0c20e1df-9eea-4ae9-aaba-462e5b4202c7 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:55.591937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0c20e1df-9eea-4ae9-aaba-462e5b4202c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:55.602077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:55.652212 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:55.659889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9d276-a3fe-4787-988e-1a818bdcfba0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:55.660348 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 251/503] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:55.661153 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20e1df-9eea-4ae9-aaba-462e5b4202c7 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:55.661445 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9d276-a3fe-4787-988e-1a818bdcfba0 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:55.759447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20e1df-9eea-4ae9-aaba-462e5b4202c7 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:55.761905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 253/504] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:55.762879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9d276-a3fe-4787-988e-1a818bdcfba0 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:55.793494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c761fe0-ed4f-470b-950c-e741b83e8397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:55.794273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c761fe0-ed4f-470b-950c-e741b83e8397 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:55.794551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c761fe0-ed4f-470b-950c-e741b83e8397 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:55.794861 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c761fe0-ed4f-470b-950c-e741b83e8397 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:55.802919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:55.805890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c761fe0-ed4f-470b-950c-e741b83e8397 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:55.807363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c761fe0-ed4f-470b-950c-e741b83e8397 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:55.825166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-0c761fe0-ed4f-470b-950c-e741b83e8397 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:55.825166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:55.825166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:55.825166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:55.825166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:55.825166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:55.825166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:55.825166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:55.825166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:55.825166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:55.825166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:55.825166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:55.825166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:59:55.825166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:59:55.825166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:55.826202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:55.827739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:59:55.827739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:59:55.827739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:55.827739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:55.827739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 252/505] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:55.827739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c761fe0-ed4f-470b-950c-e741b83e8397 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:55.864733 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9d276-a3fe-4787-988e-1a818bdcfba0 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:55.876802 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:55.898133 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e3681-083c-4e05-bdd4-531a381fb779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:55.898943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e3681-083c-4e05-bdd4-531a381fb779 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:55.899222 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e3681-083c-4e05-bdd4-531a381fb779 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:55.899498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e3681-083c-4e05-bdd4-531a381fb779 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:55.906350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e3681-083c-4e05-bdd4-531a381fb779 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:55.907802 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e3681-083c-4e05-bdd4-531a381fb779 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:55.923842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-883e3681-083c-4e05-bdd4-531a381fb779 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 08:59:55.923842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:55.923842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:55.923842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:55.923842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:55.923842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:55.923842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:55.923842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:55.923842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:55.923842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:55.923842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:55.923842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:55.923842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:59:55.923842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:59:55.923842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:59:55.923842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:55.924735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:55.925616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:55.925616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:55.925616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:59:55.925616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:59:55.925616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:59:55.925616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:59:55.925616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:59:55.925616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:59:55.925616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 08:59:55.925616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:55.925616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 254/506] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:55.926189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e3681-083c-4e05-bdd4-531a381fb779 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:55.932142 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c761fe0-ed4f-470b-950c-e741b83e8397 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:55.966783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435034e-bc3d-42fd-a352-32a69620fe34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:55.967544 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435034e-bc3d-42fd-a352-32a69620fe34 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:55.967802 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435034e-bc3d-42fd-a352-32a69620fe34 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:55.967998 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435034e-bc3d-42fd-a352-32a69620fe34 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:55.968752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435034e-bc3d-42fd-a352-32a69620fe34 None admin] RBAC: Adding query filter params (name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:55.969059 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435034e-bc3d-42fd-a352-32a69620fe34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:55.970739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435034e-bc3d-42fd-a352-32a69620fe34 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:55.979231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 253/507] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 08:59:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:55.980227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435034e-bc3d-42fd-a352-32a69620fe34 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:56.028781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:56.032349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e3681-083c-4e05-bdd4-531a381fb779 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:56.067505 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bad64a-e8e8-4f7a-ae73-947049548e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:56.068299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bad64a-e8e8-4f7a-ae73-947049548e99 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:56.068583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bad64a-e8e8-4f7a-ae73-947049548e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:56.068859 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bad64a-e8e8-4f7a-ae73-947049548e99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:56.079283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bad64a-e8e8-4f7a-ae73-947049548e99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:56.080487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435034e-bc3d-42fd-a352-32a69620fe34 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:56.080740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bad64a-e8e8-4f7a-ae73-947049548e99 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:56.091343 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:56.107908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-b3bad64a-e8e8-4f7a-ae73-947049548e99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:56.107908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:56.107908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:56.107908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:56.107908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:56.107908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:56.107908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:56.107908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:56.107908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:56.107908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:56.107908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:56.107908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:56.107908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:59:56.107908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:59:56.107908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:56.109286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:56.110440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:59:56.110440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:59:56.110440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:56.110440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:56.110440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 255/508] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:56.110440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bad64a-e8e8-4f7a-ae73-947049548e99 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:56.121175 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11b753-4ce0-4029-9f49-c25e1dcfef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:56.121973 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11b753-4ce0-4029-9f49-c25e1dcfef80 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:56.122259 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11b753-4ce0-4029-9f49-c25e1dcfef80 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:56.122544 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11b753-4ce0-4029-9f49-c25e1dcfef80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:56.134474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce11b753-4ce0-4029-9f49-c25e1dcfef80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:56.135977 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce11b753-4ce0-4029-9f49-c25e1dcfef80 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:56.153223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-ce11b753-4ce0-4029-9f49-c25e1dcfef80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:56.153223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:56.153223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:56.153223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:56.153223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:56.153223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:56.153223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:56.153223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:56.153223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:56.153223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:56.153223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:56.153223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:56.153223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:59:56.153223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:59:56.153223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:56.154249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:56.155918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:59:56.155918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:59:56.155918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:56.155918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:56.155918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 254/509] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:56.156335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11b753-4ce0-4029-9f49-c25e1dcfef80 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:56.209373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bad64a-e8e8-4f7a-ae73-947049548e99 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:56.242697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f17db-9358-49df-aecb-9e7e385b7e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:56.243353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f17db-9358-49df-aecb-9e7e385b7e32 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:56.243637 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f17db-9358-49df-aecb-9e7e385b7e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:56.243915 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f17db-9358-49df-aecb-9e7e385b7e32 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:56.244628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f17db-9358-49df-aecb-9e7e385b7e32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:56.244933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f17db-9358-49df-aecb-9e7e385b7e32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:56.246309 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f17db-9358-49df-aecb-9e7e385b7e32 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:56.254363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11b753-4ce0-4029-9f49-c25e1dcfef80 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:56.257272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 256/510] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 08:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:56.258109 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f17db-9358-49df-aecb-9e7e385b7e32 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:56.272762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:56.289377 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779c0ff-c469-4237-8558-ed3bcd095b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:56.289981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779c0ff-c469-4237-8558-ed3bcd095b0a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:56.290268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779c0ff-c469-4237-8558-ed3bcd095b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:56.290545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779c0ff-c469-4237-8558-ed3bcd095b0a None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:56.291421 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779c0ff-c469-4237-8558-ed3bcd095b0a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:56.291735 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779c0ff-c469-4237-8558-ed3bcd095b0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:56.293171 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779c0ff-c469-4237-8558-ed3bcd095b0a None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:56.330510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 255/511] 10.222.0.44 () {56 vars in 1137 bytes} [Wed Jul 20 08:59:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:56.331450 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779c0ff-c469-4237-8558-ed3bcd095b0a None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:56.353005 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:56.360047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f17db-9358-49df-aecb-9e7e385b7e32 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:56.394167 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6b68a-f01d-4b72-a174-60a1ecf56e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:56.395054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6b68a-f01d-4b72-a174-60a1ecf56e71 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:56.395343 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6b68a-f01d-4b72-a174-60a1ecf56e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:56.395624 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6b68a-f01d-4b72-a174-60a1ecf56e71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:56.406187 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6b68a-f01d-4b72-a174-60a1ecf56e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:56.407672 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6b68a-f01d-4b72-a174-60a1ecf56e71 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:56.424504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-e8f6b68a-f01d-4b72-a174-60a1ecf56e71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:56.424504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:56.424504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:56.424504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:56.424504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:56.424504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:56.424504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:56.424504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:56.424504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:56.424504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:56.424504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:56.424504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:56.424504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:59:56.424504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:59:56.424504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:56.425503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:56.426467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:59:56.426467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:59:56.426467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:56.426467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:56.426467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 257/512] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:56.426467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6b68a-f01d-4b72-a174-60a1ecf56e71 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:56.432637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779c0ff-c469-4237-8558-ed3bcd095b0a None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:56.467838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eecd75-cf56-4fe2-9f90-6d5cab6d1bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:56.468502 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eecd75-cf56-4fe2-9f90-6d5cab6d1bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:56.468879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eecd75-cf56-4fe2-9f90-6d5cab6d1bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:56.469162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eecd75-cf56-4fe2-9f90-6d5cab6d1bc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:56.469887 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eecd75-cf56-4fe2-9f90-6d5cab6d1bc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:56.470197 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eecd75-cf56-4fe2-9f90-6d5cab6d1bc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:56.471621 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eecd75-cf56-4fe2-9f90-6d5cab6d1bc6 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:56.482772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 256/513] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 08:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:56.483562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eecd75-cf56-4fe2-9f90-6d5cab6d1bc6 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:56.491646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:56.529395 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6b68a-f01d-4b72-a174-60a1ecf56e71 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:56.564540 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752ac62-354d-45c2-911e-5c465c9d6c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:56.565204 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752ac62-354d-45c2-911e-5c465c9d6c68 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:56.565490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752ac62-354d-45c2-911e-5c465c9d6c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:56.565771 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752ac62-354d-45c2-911e-5c465c9d6c68 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:56.566497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752ac62-354d-45c2-911e-5c465c9d6c68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:56.566841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752ac62-354d-45c2-911e-5c465c9d6c68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:56.568203 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752ac62-354d-45c2-911e-5c465c9d6c68 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:56.570490 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:56.580755 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 258/514] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 08:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:56.581594 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752ac62-354d-45c2-911e-5c465c9d6c68 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:56.589652 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eecd75-cf56-4fe2-9f90-6d5cab6d1bc6 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:56.626310 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e6d4b-0aba-4c69-825a-783bf4b3297a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:56.627139 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e6d4b-0aba-4c69-825a-783bf4b3297a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:56.627416 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e6d4b-0aba-4c69-825a-783bf4b3297a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:56.627696 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e6d4b-0aba-4c69-825a-783bf4b3297a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:56.635253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e6d4b-0aba-4c69-825a-783bf4b3297a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:56.636743 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e6d4b-0aba-4c69-825a-783bf4b3297a None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:56.652553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-206e6d4b-0aba-4c69-825a-783bf4b3297a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 08:59:56.652553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:56.652553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:56.652553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:56.652553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:56.652553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:56.652553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:56.652553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:56.652553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:56.652553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:56.652553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:56.652553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:56.652553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:59:56.652553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:59:56.652553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:59:56.652553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:56.653566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:56.654529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:56.654529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:56.654529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:59:56.654529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:59:56.654529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:59:56.654529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:59:56.654529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:59:56.654529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:59:56.654529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 08:59:56.654529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:56.654529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 257/515] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 08:59:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:56.655131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e6d4b-0aba-4c69-825a-783bf4b3297a None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:56.682896 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752ac62-354d-45c2-911e-5c465c9d6c68 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:56.707606 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:56.718564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4007e-3fdb-4d90-8f94-2e57e80813d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:56.719259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4007e-3fdb-4d90-8f94-2e57e80813d8 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:56.719538 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4007e-3fdb-4d90-8f94-2e57e80813d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:56.719814 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4007e-3fdb-4d90-8f94-2e57e80813d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:56.756210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e6d4b-0aba-4c69-825a-783bf4b3297a None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:56.787908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a4007e-3fdb-4d90-8f94-2e57e80813d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:56.789359 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a4007e-3fdb-4d90-8f94-2e57e80813d8 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:56.790042 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b545e7-2259-4388-866a-3fe1ff44bcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:56.790725 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b545e7-2259-4388-866a-3fe1ff44bcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:56.791013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b545e7-2259-4388-866a-3fe1ff44bcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:56.791280 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b545e7-2259-4388-866a-3fe1ff44bcd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:56.791988 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b545e7-2259-4388-866a-3fe1ff44bcd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:56.792287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b545e7-2259-4388-866a-3fe1ff44bcd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:56.793621 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b545e7-2259-4388-866a-3fe1ff44bcd4 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:56.802316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:56.811346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 258/516] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 08:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:56.811346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b545e7-2259-4388-866a-3fe1ff44bcd4 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:56.886954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 259/517] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:56.887788 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4007e-3fdb-4d90-8f94-2e57e80813d8 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:56.908454 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b545e7-2259-4388-866a-3fe1ff44bcd4 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:56.911925 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:56.942555 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d04a68-9efe-48ee-aa95-4bf48a1ce464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:56.943251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d04a68-9efe-48ee-aa95-4bf48a1ce464 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:56.943536 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d04a68-9efe-48ee-aa95-4bf48a1ce464 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:56.943812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d04a68-9efe-48ee-aa95-4bf48a1ce464 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:56.944775 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d04a68-9efe-48ee-aa95-4bf48a1ce464 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:56.946206 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d04a68-9efe-48ee-aa95-4bf48a1ce464 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:56.964671 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 259/518] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 08:59:56 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:59:56.965469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d04a68-9efe-48ee-aa95-4bf48a1ce464 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:56.987143 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4007e-3fdb-4d90-8f94-2e57e80813d8 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:57.027144 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa3ced-6d45-45f0-8b5c-cb338e722ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:57.027954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa3ced-6d45-45f0-8b5c-cb338e722ded None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:57.028249 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa3ced-6d45-45f0-8b5c-cb338e722ded None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:57.028521 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa3ced-6d45-45f0-8b5c-cb338e722ded None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:57.036701 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa3ced-6d45-45f0-8b5c-cb338e722ded None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:57.038193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa3ced-6d45-45f0-8b5c-cb338e722ded None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:57.054073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-fbaa3ced-6d45-45f0-8b5c-cb338e722ded None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:59:57.054073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:57.054073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:57.054073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:57.054073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:57.054073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:57.054073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:57.054073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:57.054073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:57.054073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:57.054073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:57.054073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:57.054073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:59:57.054073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:59:57.054073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:57.055191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:57.056086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:59:57.056086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:59:57.056086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:59:57.056086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:59:57.056086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:59:57.056086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:59:57.056086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:59:57.056086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:57.056086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 260/519] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:57.056086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa3ced-6d45-45f0-8b5c-cb338e722ded None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:57.057305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:57.116266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d04a68-9efe-48ee-aa95-4bf48a1ce464 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:57.159127 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f672f4-b49b-4f30-a437-380a39984be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:57.159968 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f672f4-b49b-4f30-a437-380a39984be3 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:57.160398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f672f4-b49b-4f30-a437-380a39984be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:57.160718 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f672f4-b49b-4f30-a437-380a39984be3 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:57.161674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f672f4-b49b-4f30-a437-380a39984be3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:57.162079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f672f4-b49b-4f30-a437-380a39984be3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:57.163639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f672f4-b49b-4f30-a437-380a39984be3 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:57.171724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:57.188026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 260/520] 10.222.0.44 () {56 vars in 1139 bytes} [Wed Jul 20 08:59:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:57.188271 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa3ced-6d45-45f0-8b5c-cb338e722ded None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:57.189191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f672f4-b49b-4f30-a437-380a39984be3 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:57.225726 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3d798-3952-4a82-94a5-7e74dc2a7246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:57.226652 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3d798-3952-4a82-94a5-7e74dc2a7246 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:57.226963 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3d798-3952-4a82-94a5-7e74dc2a7246 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:57.227238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3d798-3952-4a82-94a5-7e74dc2a7246 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:57.235839 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3d798-3952-4a82-94a5-7e74dc2a7246 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:57.237280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3d798-3952-4a82-94a5-7e74dc2a7246 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:57.251885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-7cf3d798-3952-4a82-94a5-7e74dc2a7246 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 08:59:57.251885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:57.251885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:57.251885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:57.251885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:57.251885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:57.251885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:57.251885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:57.251885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:57.251885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:57.251885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:57.251885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:57.251885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:59:57.251885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:59:57.251885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:59:57.251885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:57.252964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:57.254003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:57.254003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:57.254003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:59:57.254003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:59:57.254003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:59:57.254003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:59:57.254003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:59:57.254003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:59:57.254003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 08:59:57.254003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:57.254003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 261/521] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 08:59:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:57.254003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3d798-3952-4a82-94a5-7e74dc2a7246 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:57.298717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f672f4-b49b-4f30-a437-380a39984be3 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:57.314183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:57.338083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5266d2-d2c1-408e-b416-fb69dbee0307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:57.339065 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5266d2-d2c1-408e-b416-fb69dbee0307 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:57.339447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5266d2-d2c1-408e-b416-fb69dbee0307 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:57.340126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5266d2-d2c1-408e-b416-fb69dbee0307 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:57.350614 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5266d2-d2c1-408e-b416-fb69dbee0307 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:57.352207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5266d2-d2c1-408e-b416-fb69dbee0307 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:57.363070 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3d798-3952-4a82-94a5-7e74dc2a7246 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:57.371535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-0f5266d2-d2c1-408e-b416-fb69dbee0307 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:57.371535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:57.371535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:57.371535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:57.371535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:57.371535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:57.371535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:57.371535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:57.371535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:57.371535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:57.371535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:57.371535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:57.371535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:59:57.371535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:59:57.371535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:57.372572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:57.373759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:59:57.373759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:59:57.373759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:57.373759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:57.373759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 261/522] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:57.374050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5266d2-d2c1-408e-b416-fb69dbee0307 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:57.404430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5d129-71f1-4ad5-a212-89f71a34ac6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:57.405102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5d129-71f1-4ad5-a212-89f71a34ac6a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:57.405395 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5d129-71f1-4ad5-a212-89f71a34ac6a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:57.405681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5d129-71f1-4ad5-a212-89f71a34ac6a None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:57.406470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5d129-71f1-4ad5-a212-89f71a34ac6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:57.406811 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5d129-71f1-4ad5-a212-89f71a34ac6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:57.408272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b5d129-71f1-4ad5-a212-89f71a34ac6a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:57.431599 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 262/523] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 08:59:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:57.431599 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5d129-71f1-4ad5-a212-89f71a34ac6a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:57.433539 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:57.483470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5266d2-d2c1-408e-b416-fb69dbee0307 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:57.517327 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4242b-fa46-4a73-b571-8eda5f11bcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:57.517980 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4242b-fa46-4a73-b571-8eda5f11bcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:57.518256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4242b-fa46-4a73-b571-8eda5f11bcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:57.518528 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4242b-fa46-4a73-b571-8eda5f11bcf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:57.537795 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5d129-71f1-4ad5-a212-89f71a34ac6a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:57.561331 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:57.579328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a99f3e-21e0-4adb-965d-b4e33a92ccac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:57.579935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a99f3e-21e0-4adb-965d-b4e33a92ccac None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:57.580221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a99f3e-21e0-4adb-965d-b4e33a92ccac None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:57.580504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a99f3e-21e0-4adb-965d-b4e33a92ccac None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:57.581345 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a99f3e-21e0-4adb-965d-b4e33a92ccac None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:57.581661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a99f3e-21e0-4adb-965d-b4e33a92ccac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:57.583123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a99f3e-21e0-4adb-965d-b4e33a92ccac None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:57.598354 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4242b-fa46-4a73-b571-8eda5f11bcf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:57.599946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4242b-fa46-4a73-b571-8eda5f11bcf0 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:57.602142 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 263/524] 10.222.0.44 () {56 vars in 1137 bytes} [Wed Jul 20 08:59:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:57.608243 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a99f3e-21e0-4adb-965d-b4e33a92ccac None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:57.678522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:57.704065 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 262/525] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:57.705167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4242b-fa46-4a73-b571-8eda5f11bcf0 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:57.726887 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a99f3e-21e0-4adb-965d-b4e33a92ccac None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:57.763119 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176de80-d3bf-46e2-bc6d-86d64afb980d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:57.763951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176de80-d3bf-46e2-bc6d-86d64afb980d None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:57.764238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176de80-d3bf-46e2-bc6d-86d64afb980d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:57.764518 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176de80-d3bf-46e2-bc6d-86d64afb980d None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:57.765241 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176de80-d3bf-46e2-bc6d-86d64afb980d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:57.765786 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176de80-d3bf-46e2-bc6d-86d64afb980d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:57.766999 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176de80-d3bf-46e2-bc6d-86d64afb980d None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:57.773464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:57.779095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 264/526] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 08:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:57.781327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176de80-d3bf-46e2-bc6d-86d64afb980d None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:57.813776 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4242b-fa46-4a73-b571-8eda5f11bcf0 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:57.849933 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796a91e-5151-4336-aa34-464d0c61872e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:57.850633 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796a91e-5151-4336-aa34-464d0c61872e None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:57.850956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796a91e-5151-4336-aa34-464d0c61872e None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:57.851280 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1796a91e-5151-4336-aa34-464d0c61872e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:57.852116 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796a91e-5151-4336-aa34-464d0c61872e None admin] RBAC: Adding query filter params (role.id=ae949d8bea4b40d9b1a9fa207d539663, scope.project.id=a498284274454d25b1b118a88996e522, user.id=be08c43f0d0c4a8194596e54500b44ed) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:57.852439 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796a91e-5151-4336-aa34-464d0c61872e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:57.853891 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1796a91e-5151-4336-aa34-464d0c61872e None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:57.870724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 263/527] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:57 2022] GET /identity/v3/role_assignments?user.id=be08c43f0d0c4a8194596e54500b44ed&scope.project.id=a498284274454d25b1b118a88996e522&role.id=ae949d8bea4b40d9b1a9fa207d539663 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:57.870724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796a91e-5151-4336-aa34-464d0c61872e None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:57.904859 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176de80-d3bf-46e2-bc6d-86d64afb980d None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:57.941332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:57.959325 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e5727-99e4-4b14-bf29-8acb5c2d77c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:57.960611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e5727-99e4-4b14-bf29-8acb5c2d77c3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:57.960996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e5727-99e4-4b14-bf29-8acb5c2d77c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:57.961839 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e5727-99e4-4b14-bf29-8acb5c2d77c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:58.018230 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1796a91e-5151-4336-aa34-464d0c61872e None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:58.052149 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e5727-99e4-4b14-bf29-8acb5c2d77c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:58.052853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:58.055111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5e5727-99e4-4b14-bf29-8acb5c2d77c3 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:58.061751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6e59d-e222-4a74-9ab9-281cc708fbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:58.061751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6e59d-e222-4a74-9ab9-281cc708fbfe None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:58.061751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6e59d-e222-4a74-9ab9-281cc708fbfe None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:58.062078 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6e59d-e222-4a74-9ab9-281cc708fbfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:58.074879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6e59d-e222-4a74-9ab9-281cc708fbfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:58.077443 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6e59d-e222-4a74-9ab9-281cc708fbfe None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:58.100689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-7aa6e59d-e222-4a74-9ab9-281cc708fbfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:58.100689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:58.100689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:58.100689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:58.100689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:58.100689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:58.100689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:58.100689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:58.100689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:58.100689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:58.100689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:58.100689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:58.100689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:59:58.100689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:59:58.100689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:58.101603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:58.102473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:59:58.102473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:59:58.102473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:58.102473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:58.102473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 264/528] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:58.102884 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6e59d-e222-4a74-9ab9-281cc708fbfe None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:58.182470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 265/529] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 08:59:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:59:58.182470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e5727-99e4-4b14-bf29-8acb5c2d77c3 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:58.206081 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:58.254557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6e59d-e222-4a74-9ab9-281cc708fbfe None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:58.295876 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e5727-99e4-4b14-bf29-8acb5c2d77c3 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:58.298989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe50ed-7b5e-486e-96f4-64363c7caa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:58.299761 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe50ed-7b5e-486e-96f4-64363c7caa6a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:58.300041 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe50ed-7b5e-486e-96f4-64363c7caa6a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:58.300315 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe50ed-7b5e-486e-96f4-64363c7caa6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:58.307361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe50ed-7b5e-486e-96f4-64363c7caa6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:58.308908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe50ed-7b5e-486e-96f4-64363c7caa6a None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:58.325513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-4cbe50ed-7b5e-486e-96f4-64363c7caa6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:59:58.325513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:58.325513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:58.325513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:58.325513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:58.325513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:58.325513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:58.325513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:58.325513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:58.325513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:58.325513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:58.325513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:58.325513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:59:58.325513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:59:58.325513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:58.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:58.327789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:59:58.327789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:59:58.327789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:59:58.327789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:59:58.327789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:59:58.327789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:59:58.327789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:59:58.327789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:58.327789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 265/530] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:58.327789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe50ed-7b5e-486e-96f4-64363c7caa6a None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:58.342986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e999921-3457-41ef-bb4d-e160e461ba1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:58.343973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e999921-3457-41ef-bb4d-e160e461ba1b None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:58.344407 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e999921-3457-41ef-bb4d-e160e461ba1b None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:58.344836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e999921-3457-41ef-bb4d-e160e461ba1b None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:58.345729 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e999921-3457-41ef-bb4d-e160e461ba1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:58.346281 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e999921-3457-41ef-bb4d-e160e461ba1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:58.347935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e999921-3457-41ef-bb4d-e160e461ba1b None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:58.366017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 266/531] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 08:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:58.367127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e999921-3457-41ef-bb4d-e160e461ba1b None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:58.374081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:58.433796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe50ed-7b5e-486e-96f4-64363c7caa6a None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:58.460613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:58.471778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a18186-0fb4-4d7b-9a48-2d69d6b53aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:58.472826 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a18186-0fb4-4d7b-9a48-2d69d6b53aba None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:58.473251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a18186-0fb4-4d7b-9a48-2d69d6b53aba None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:58.473695 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a18186-0fb4-4d7b-9a48-2d69d6b53aba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:58.481409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e999921-3457-41ef-bb4d-e160e461ba1b None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:58.485616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a18186-0fb4-4d7b-9a48-2d69d6b53aba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:58.487262 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a18186-0fb4-4d7b-9a48-2d69d6b53aba None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:58.506611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-e5a18186-0fb4-4d7b-9a48-2d69d6b53aba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:58.506611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:58.506611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:58.506611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:58.506611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:58.506611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:58.506611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:58.506611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:58.506611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:58.506611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:58.506611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:58.506611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:58.506611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:59:58.506611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:59:58.506611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:58.507700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:58.508690 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:59:58.508690 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:59:58.508690 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:59:58.508690 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:58.510631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 266/532] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:58.511551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a18186-0fb4-4d7b-9a48-2d69d6b53aba None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:58.523217 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5d4a3-1f4b-46bf-9eb2-2fb4d0ca8fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:58.523889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5d4a3-1f4b-46bf-9eb2-2fb4d0ca8fef None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:58.524179 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5d4a3-1f4b-46bf-9eb2-2fb4d0ca8fef None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:58.524460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5d4a3-1f4b-46bf-9eb2-2fb4d0ca8fef None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:58.525376 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b5d4a3-1f4b-46bf-9eb2-2fb4d0ca8fef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:58.525694 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b5d4a3-1f4b-46bf-9eb2-2fb4d0ca8fef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:58.527294 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b5d4a3-1f4b-46bf-9eb2-2fb4d0ca8fef None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:58.541929 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 267/533] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 08:59:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:58.547746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5d4a3-1f4b-46bf-9eb2-2fb4d0ca8fef None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:58.613803 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:58.621459 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a18186-0fb4-4d7b-9a48-2d69d6b53aba None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:58.659447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf158a4-5b34-4210-a217-f432a2c3c2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:58.660483 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf158a4-5b34-4210-a217-f432a2c3c2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:58.660905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf158a4-5b34-4210-a217-f432a2c3c2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:58.661325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf158a4-5b34-4210-a217-f432a2c3c2d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:58.661458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5d4a3-1f4b-46bf-9eb2-2fb4d0ca8fef None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:58.668720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf158a4-5b34-4210-a217-f432a2c3c2d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:58.670874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf158a4-5b34-4210-a217-f432a2c3c2d9 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:58.686374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-edf158a4-5b34-4210-a217-f432a2c3c2d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:59:58.686374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:58.686374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:58.686374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:58.686374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:58.686374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:58.686374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:58.686374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:58.686374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:58.686374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:58.686374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:58.686374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:58.686374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:59:58.686374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:59:58.686374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:58.687534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:58.688556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:59:58.688556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:59:58.688556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:59:58.688556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:59:58.688556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:59:58.688556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:59:58.688556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:59:58.688556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:58.695556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 267/534] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:58.696688 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf158a4-5b34-4210-a217-f432a2c3c2d9 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:58.699032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c35636-4361-4a83-b3f4-fd7a39efc800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:58.699737 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c35636-4361-4a83-b3f4-fd7a39efc800 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:58.700172 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c35636-4361-4a83-b3f4-fd7a39efc800 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:58.700457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c35636-4361-4a83-b3f4-fd7a39efc800 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:58.701779 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c35636-4361-4a83-b3f4-fd7a39efc800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:58.702340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c35636-4361-4a83-b3f4-fd7a39efc800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:58.704006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c35636-4361-4a83-b3f4-fd7a39efc800 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:58.721626 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 268/535] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 08:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:58.722531 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c35636-4361-4a83-b3f4-fd7a39efc800 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:58.725720 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:58.806395 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf158a4-5b34-4210-a217-f432a2c3c2d9 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:58.837200 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c35636-4361-4a83-b3f4-fd7a39efc800 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:58.843455 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8eebb-215a-4eb1-ad04-a94e22859547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:58.844392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8eebb-215a-4eb1-ad04-a94e22859547 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:58.844818 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8eebb-215a-4eb1-ad04-a94e22859547 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:58.845237 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8eebb-215a-4eb1-ad04-a94e22859547 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:58.846268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8eebb-215a-4eb1-ad04-a94e22859547 None admin] RBAC: Adding query filter params (role.id=ae949d8bea4b40d9b1a9fa207d539663, scope.project.id=a498284274454d25b1b118a88996e522, user.id=bb93eb1d25c4410591125b0ab7a73149) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:58.846814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8eebb-215a-4eb1-ad04-a94e22859547 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:58.848500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec8eebb-215a-4eb1-ad04-a94e22859547 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:58.854828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 268/536] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:58 2022] GET /identity/v3/role_assignments?user.id=bb93eb1d25c4410591125b0ab7a73149&scope.project.id=a498284274454d25b1b118a88996e522&role.id=ae949d8bea4b40d9b1a9fa207d539663 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:58.855780 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8eebb-215a-4eb1-ad04-a94e22859547 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:58.880693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:58.897770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4916af9-3b44-46e3-98f4-9fae0187279b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:58.898724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4916af9-3b44-46e3-98f4-9fae0187279b None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:58.899181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4916af9-3b44-46e3-98f4-9fae0187279b None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:58.899608 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4916af9-3b44-46e3-98f4-9fae0187279b None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:58.900731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4916af9-3b44-46e3-98f4-9fae0187279b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:58.901205 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4916af9-3b44-46e3-98f4-9fae0187279b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:58.907252 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4916af9-3b44-46e3-98f4-9fae0187279b None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:58.929019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 269/537] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 08:59:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:58.934004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4916af9-3b44-46e3-98f4-9fae0187279b None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:58.971506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8eebb-215a-4eb1-ad04-a94e22859547 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:59.009500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba485f1-4cd6-4cb8-96c2-da2ba112c79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:59.009817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba485f1-4cd6-4cb8-96c2-da2ba112c79c None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:59.010340 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba485f1-4cd6-4cb8-96c2-da2ba112c79c None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:59.010712 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba485f1-4cd6-4cb8-96c2-da2ba112c79c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:59.022073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba485f1-4cd6-4cb8-96c2-da2ba112c79c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:59.022073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba485f1-4cd6-4cb8-96c2-da2ba112c79c None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:59.036935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-5ba485f1-4cd6-4cb8-96c2-da2ba112c79c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:59:59.036935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:59.036935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:59.036935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:59.036935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:59.036935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:59.036935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:59.036935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:59.036935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:59.036935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:59.036935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:59.036935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:59.036935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:59:59.036935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:59:59.036935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:59.038160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:59.039275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:59:59.039275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:59:59.039275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:59:59.039275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:59:59.039275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:59:59.039275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:59:59.039275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:59:59.039275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 08:59:59.042723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 269/538] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 08:59:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:59.055198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:59.056622 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba485f1-4cd6-4cb8-96c2-da2ba112c79c None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:59.101248 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4916af9-3b44-46e3-98f4-9fae0187279b None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:59.144030 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69479f43-238f-4b4b-af8f-68ba367007d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:59.144622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69479f43-238f-4b4b-af8f-68ba367007d3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:59.144906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69479f43-238f-4b4b-af8f-68ba367007d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:59.145183 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69479f43-238f-4b4b-af8f-68ba367007d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:59.146044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69479f43-238f-4b4b-af8f-68ba367007d3 None admin] RBAC: Adding query filter params (role.id=ae949d8bea4b40d9b1a9fa207d539663, scope.project.id=a498284274454d25b1b118a88996e522, user.id=5e5ad4e122ae471cad4cf1b0e3111dd9) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:59.146355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69479f43-238f-4b4b-af8f-68ba367007d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:59.147918 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69479f43-238f-4b4b-af8f-68ba367007d3 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:59.157290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 270/539] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:58 2022] GET /identity/v3/role_assignments?user.id=5e5ad4e122ae471cad4cf1b0e3111dd9&scope.project.id=a498284274454d25b1b118a88996e522&role.id=ae949d8bea4b40d9b1a9fa207d539663 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:59.158870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f852af-10d1-4d1c-998b-fc143e36d7af None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:59.159160 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f852af-10d1-4d1c-998b-fc143e36d7af None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:59.159447 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f852af-10d1-4d1c-998b-fc143e36d7af None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:59.166055 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 271/540] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 08:59:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:59:59.168221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adb0fc-2ae5-46f4-ac5b-1a85f227d4e4 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:59.168521 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adb0fc-2ae5-46f4-ac5b-1a85f227d4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:59.168870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adb0fc-2ae5-46f4-ac5b-1a85f227d4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:59.184788 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:59.195186 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-89adb0fc-2ae5-46f4-ac5b-1a85f227d4e4 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:59:59.196495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-89adb0fc-2ae5-46f4-ac5b-1a85f227d4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:59:59.218857 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba485f1-4cd6-4cb8-96c2-da2ba112c79c None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:59.265124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce157a11-5af9-4b09-b85e-8cc860a8043b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:59.266335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce157a11-5af9-4b09-b85e-8cc860a8043b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:59.266655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce157a11-5af9-4b09-b85e-8cc860a8043b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:59.267025 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce157a11-5af9-4b09-b85e-8cc860a8043b None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:59.268129 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce157a11-5af9-4b09-b85e-8cc860a8043b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:59.268450 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce157a11-5af9-4b09-b85e-8cc860a8043b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:59.270260 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce157a11-5af9-4b09-b85e-8cc860a8043b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:59.289028 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 272/541] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 08:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:59:59.309172 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 270/542] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 08:59:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:59.314951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89adb0fc-2ae5-46f4-ac5b-1a85f227d4e4 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:59.315764 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce157a11-5af9-4b09-b85e-8cc860a8043b None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:59.411646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:59.486329 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89adb0fc-2ae5-46f4-ac5b-1a85f227d4e4 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:59.498879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce157a11-5af9-4b09-b85e-8cc860a8043b None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:59.559724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85f4c6-7501-4561-b61d-5279babe20ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:59.561051 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85f4c6-7501-4561-b61d-5279babe20ef None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:59.561342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85f4c6-7501-4561-b61d-5279babe20ef None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:59.561625 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85f4c6-7501-4561-b61d-5279babe20ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:59.565070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f07bb-3740-456c-a6ec-5fb62107d100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:59.565733 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f07bb-3740-456c-a6ec-5fb62107d100 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:59.566017 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f07bb-3740-456c-a6ec-5fb62107d100 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:59.566294 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f07bb-3740-456c-a6ec-5fb62107d100 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:59.567174 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f07bb-3740-456c-a6ec-5fb62107d100 None admin] RBAC: Adding query filter params (role.id=ae949d8bea4b40d9b1a9fa207d539663, scope.project.id=a498284274454d25b1b118a88996e522, user.id=99dc3db93e9045b8b3ef59a030822972) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:59.567490 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f07bb-3740-456c-a6ec-5fb62107d100 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:59.569886 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95f07bb-3740-456c-a6ec-5fb62107d100 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:59.581169 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 271/543] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 08:59:59 2022] GET /identity/v3/role_assignments?user.id=99dc3db93e9045b8b3ef59a030822972&scope.project.id=a498284274454d25b1b118a88996e522&role.id=ae949d8bea4b40d9b1a9fa207d539663 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:59:59.595995 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85f4c6-7501-4561-b61d-5279babe20ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:59:59.604072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-7a85f4c6-7501-4561-b61d-5279babe20ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:59:59.604072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:59:59.604072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:59:59.604072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:59:59.604072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:59:59.604072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:59:59.604072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:59:59.604072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:59:59.604072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:59:59.604072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:59:59.604072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:59:59.604072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:59:59.604072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:59:59.604072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:59:59.604072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:59:59.604072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:59:59.605032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:59:59.605032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:59:59.605032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:59:59.605032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:59:59.605032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:59:59.605032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:59:59.605032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:59:59.605032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:59:59.605032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:59:59.605032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:59:59.605032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:59:59.605032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 08:59:59.605032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:59:59.605032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:59:59.605032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:59:59.605032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:59:59.605821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:59:59.605821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:59:59.605821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:59:59.605821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:59:59.605821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:59:59.605821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:59:59.605821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:59:59.605821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:59:59.605821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:59:59.605821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:59:59.605821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:59:59.605821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 08:59:59.611280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 273/544] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 08:59:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:59:59.617203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f07bb-3740-456c-a6ec-5fb62107d100 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:59.625536 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:59:59.787778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f07bb-3740-456c-a6ec-5fb62107d100 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:59:59.835387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee0ff1-4e19-4e28-af84-ff153b740638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:59:59.836069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee0ff1-4e19-4e28-af84-ff153b740638 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:59:59.836352 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee0ff1-4e19-4e28-af84-ff153b740638 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:59:59.836625 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee0ff1-4e19-4e28-af84-ff153b740638 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:59:59.837373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee0ff1-4e19-4e28-af84-ff153b740638 None admin] RBAC: Adding query filter params (name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:59:59.837677 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee0ff1-4e19-4e28-af84-ff153b740638 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:59:59.839065 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee0ff1-4e19-4e28-af84-ff153b740638 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:59:59.847770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 272/545] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 08:59:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:59:59.853231 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85f4c6-7501-4561-b61d-5279babe20ef None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:59:59.984263 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85f4c6-7501-4561-b61d-5279babe20ef None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:00.009492 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:00.039649 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd53f96-e064-478e-abd7-58d994649ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:00.040376 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd53f96-e064-478e-abd7-58d994649ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:00.040659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd53f96-e064-478e-abd7-58d994649ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:00.040940 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd53f96-e064-478e-abd7-58d994649ca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:00.056098 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd53f96-e064-478e-abd7-58d994649ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:00.057552 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd53f96-e064-478e-abd7-58d994649ca8 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:00.080632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-6fd53f96-e064-478e-abd7-58d994649ca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:00.080632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:00.080632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:00.080632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:00.080632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:00.080632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:00.080632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:00.080632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:00.080632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:00.080632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:00.080632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:00.080632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:00.080632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:00.080632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:00.080632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:00.081495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:00.082324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:00.082324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:00.082324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:00.082324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:00.082324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 274/546] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 08:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:00.087352 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd53f96-e064-478e-abd7-58d994649ca8 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:00.087900 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86404fc8-a729-4221-af09-7abf168cb8a6 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:00.088441 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86404fc8-a729-4221-af09-7abf168cb8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:00.088833 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86404fc8-a729-4221-af09-7abf168cb8a6 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:00.090386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 273/547] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:00.101360 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a495e-c00c-4106-b9ad-cbd41a45ac44 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:00.101732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a495e-c00c-4106-b9ad-cbd41a45ac44 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:00.102024 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a495e-c00c-4106-b9ad-cbd41a45ac44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:00.138456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-0d3a495e-c00c-4106-b9ad-cbd41a45ac44 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:00.141700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0d3a495e-c00c-4106-b9ad-cbd41a45ac44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:00.233617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:00.242338 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd53f96-e064-478e-abd7-58d994649ca8 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:00.256147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 274/548] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:00.280359 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f7c74-71b2-4a83-8163-4824a351c377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:00.281320 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f7c74-71b2-4a83-8163-4824a351c377 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:00.281778 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f7c74-71b2-4a83-8163-4824a351c377 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:00.282258 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f7c74-71b2-4a83-8163-4824a351c377 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:00.283198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f7c74-71b2-4a83-8163-4824a351c377 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:00.283682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f7c74-71b2-4a83-8163-4824a351c377 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:00.285301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f7c74-71b2-4a83-8163-4824a351c377 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:00.300477 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 275/549] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:00.302731 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a495e-c00c-4106-b9ad-cbd41a45ac44 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:00.306143 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f7c74-71b2-4a83-8163-4824a351c377 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:00.339819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:00.418702 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f7c74-71b2-4a83-8163-4824a351c377 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:00.457484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbe481-f00f-470d-9e71-b55a772a340d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:00.458385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbe481-f00f-470d-9e71-b55a772a340d None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:00.458769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbe481-f00f-470d-9e71-b55a772a340d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:00.459123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbe481-f00f-470d-9e71-b55a772a340d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:00.468087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbe481-f00f-470d-9e71-b55a772a340d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:00.470483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbe481-f00f-470d-9e71-b55a772a340d None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:00.486502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-4cbbe481-f00f-470d-9e71-b55a772a340d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:00:00.486502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:00.486502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:00.486502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:00.486502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:00.486502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:00.486502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:00.486502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:00.486502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:00.486502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:00.486502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:00.486502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:00.486502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:00.486502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:00.486502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:00.486502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:00.488166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:00.489418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:00.489418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:00.489418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:00.489418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:00.489418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:00.489418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:00.489418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:00.489418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:00.489418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:00:00.489418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:00.489943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 276/550] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:00:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:00.494798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbe481-f00f-470d-9e71-b55a772a340d None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:00.603066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:00.603922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a495e-c00c-4106-b9ad-cbd41a45ac44 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:00.609649 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbe481-f00f-470d-9e71-b55a772a340d None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:00.647627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05084325-ca33-4469-86b6-29f0508c9b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:00.648297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05084325-ca33-4469-86b6-29f0508c9b62 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:00.648580 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05084325-ca33-4469-86b6-29f0508c9b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:00.648855 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05084325-ca33-4469-86b6-29f0508c9b62 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:00.649692 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05084325-ca33-4469-86b6-29f0508c9b62 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:00.649996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05084325-ca33-4469-86b6-29f0508c9b62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:00.651441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05084325-ca33-4469-86b6-29f0508c9b62 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:00.671160 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 277/551] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Jul 20 09:00:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:00.676607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05084325-ca33-4469-86b6-29f0508c9b62 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:00.701684 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd44a7-1d20-48d5-ba5a-e8ab3e24a71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:00.702503 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd44a7-1d20-48d5-ba5a-e8ab3e24a71d None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:00.702817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd44a7-1d20-48d5-ba5a-e8ab3e24a71d None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:00.703098 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd44a7-1d20-48d5-ba5a-e8ab3e24a71d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:00.742113 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd44a7-1d20-48d5-ba5a-e8ab3e24a71d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:00.750605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-04dd44a7-1d20-48d5-ba5a-e8ab3e24a71d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:00.750605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:00.750605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:00.750605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:00.750605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:00.750605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:00.750605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:00.750605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:00.750605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:00.750605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:00.750605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:00.750605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:00.750605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:00.750605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:00.750605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:00.750605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:00.751736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:00.751736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:00.751736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:00.751736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:00.751736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:00.751736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:00.751736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:00.751736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:00.751736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:00.751736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:00.751736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:00.751736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:00.751736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:00.751736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:00.751736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:00.751736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:00.752918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:00.752918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:00.752918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:00.752918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:00.752918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:00.752918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:00.752918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:00.752918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:00.752918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:00.752918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:00.752918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:00.752918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:00.760300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 275/552] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:00:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 460 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:00.767753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd44a7-1d20-48d5-ba5a-e8ab3e24a71d None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:00.784011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05084325-ca33-4469-86b6-29f0508c9b62 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:00.821774 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e3f0d-c576-4aee-8a1c-691a58a9a8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:00.822758 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e3f0d-c576-4aee-8a1c-691a58a9a8fd None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:00.823044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e3f0d-c576-4aee-8a1c-691a58a9a8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:00.823320 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587e3f0d-c576-4aee-8a1c-691a58a9a8fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:00.859095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:00.903972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e3f0d-c576-4aee-8a1c-691a58a9a8fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:00.905769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587e3f0d-c576-4aee-8a1c-691a58a9a8fd None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:00.943843 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:01.015524 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 278/553] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:00:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 09:00:01.020949 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e3f0d-c576-4aee-8a1c-691a58a9a8fd None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:01.087808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd44a7-1d20-48d5-ba5a-e8ab3e24a71d None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:01.124319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:01.145117 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca59128-b024-4673-85c2-952e6470b2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:01.146207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca59128-b024-4673-85c2-952e6470b2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:01.147452 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca59128-b024-4673-85c2-952e6470b2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:01.148496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca59128-b024-4673-85c2-952e6470b2e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:01.149409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca59128-b024-4673-85c2-952e6470b2e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:01.150027 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca59128-b024-4673-85c2-952e6470b2e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:01.152527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca59128-b024-4673-85c2-952e6470b2e3 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:01.169185 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 276/554] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 09:00:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:01.178346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca59128-b024-4673-85c2-952e6470b2e3 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:01.249822 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587e3f0d-c576-4aee-8a1c-691a58a9a8fd None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:01.290808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca59128-b024-4673-85c2-952e6470b2e3 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:01.308458 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:01.332218 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fddb63-dbda-46dc-8793-c1a189d4fdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:01.333690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb1715-22b2-4357-b2ac-0c6059d3b135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:01.334663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fddb63-dbda-46dc-8793-c1a189d4fdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:01.335670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb1715-22b2-4357-b2ac-0c6059d3b135 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:01.336105 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fddb63-dbda-46dc-8793-c1a189d4fdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:01.336622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb1715-22b2-4357-b2ac-0c6059d3b135 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:01.337122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fddb63-dbda-46dc-8793-c1a189d4fdd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:01.337538 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb1715-22b2-4357-b2ac-0c6059d3b135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:01.353793 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fddb63-dbda-46dc-8793-c1a189d4fdd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:01.355334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb1715-22b2-4357-b2ac-0c6059d3b135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:01.355727 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fddb63-dbda-46dc-8793-c1a189d4fdd4 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:01.356837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb1715-22b2-4357-b2ac-0c6059d3b135 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:01.374126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-f6fddb63-dbda-46dc-8793-c1a189d4fdd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:01.374126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:01.374126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:01.374126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:01.374126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:01.374126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:01.374126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:01.374126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:01.374126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:01.374126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:01.374126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:01.374126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:01.374126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:01.374126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:01.374126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:01.375265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:01.376344 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:01.376344 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:01.376344 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:01.376344 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:01.377084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-d2cb1715-22b2-4357-b2ac-0c6059d3b135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:01.377084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:01.377084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:01.377084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:01.377084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:01.377084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:01.377084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:01.377084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:01.377084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:01.377084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:01.377084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:01.377084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:01.377084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:01.377084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:01.377084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:01.377894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:01.379377 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:01.379377 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:01.379377 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:01.379377 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:01.380725 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 277/556] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:01.382251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc615db-255e-404a-b6ca-7a6e36f18d89 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:01.382901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 279/556] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:01.383241 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc615db-255e-404a-b6ca-7a6e36f18d89 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:01.383702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc615db-255e-404a-b6ca-7a6e36f18d89 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:01.384160 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb1715-22b2-4357-b2ac-0c6059d3b135 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:01.385670 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 278/557] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:01.386596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc615db-255e-404a-b6ca-7a6e36f18d89 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:01.494191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc615db-255e-404a-b6ca-7a6e36f18d89 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:01.495672 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:01.501069 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb1715-22b2-4357-b2ac-0c6059d3b135 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:01.532177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21eadd0-57ad-4d62-ad54-bdbca25c4e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:01.533234 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eadd0-57ad-4d62-ad54-bdbca25c4e61 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:01.533730 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eadd0-57ad-4d62-ad54-bdbca25c4e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:01.534232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eadd0-57ad-4d62-ad54-bdbca25c4e61 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:01.537783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9e1af-0b8c-4838-b7a3-4ba695be79ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:01.538081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21eadd0-57ad-4d62-ad54-bdbca25c4e61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:01.538648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21eadd0-57ad-4d62-ad54-bdbca25c4e61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:01.540394 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21eadd0-57ad-4d62-ad54-bdbca25c4e61 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:01.540582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9e1af-0b8c-4838-b7a3-4ba695be79ab None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:01.540877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9e1af-0b8c-4838-b7a3-4ba695be79ab None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:01.541161 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9e1af-0b8c-4838-b7a3-4ba695be79ab None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:01.541955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9e1af-0b8c-4838-b7a3-4ba695be79ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:01.542267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9e1af-0b8c-4838-b7a3-4ba695be79ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:01.543635 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9e1af-0b8c-4838-b7a3-4ba695be79ab None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:01.557600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 279/558] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:01.559266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e746e-cce5-46d4-8f37-690aa8162cf4 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:01.559772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e746e-cce5-46d4-8f37-690aa8162cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:01.560259 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e746e-cce5-46d4-8f37-690aa8162cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:01.563716 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 280/559] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:01.564660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9e1af-0b8c-4838-b7a3-4ba695be79ab None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:01.580227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-120e746e-cce5-46d4-8f37-690aa8162cf4 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:01.581755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-120e746e-cce5-46d4-8f37-690aa8162cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:01.607143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:01.662740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 280/560] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:01.663963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e746e-cce5-46d4-8f37-690aa8162cf4 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:01.687600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9e1af-0b8c-4838-b7a3-4ba695be79ab None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:01.732007 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3588972-cea8-4baf-9bc3-01082d9dc9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:01.732947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3588972-cea8-4baf-9bc3-01082d9dc9bf None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:01.733217 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3588972-cea8-4baf-9bc3-01082d9dc9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:01.733497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3588972-cea8-4baf-9bc3-01082d9dc9bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:01.741913 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3588972-cea8-4baf-9bc3-01082d9dc9bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:01.743438 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3588972-cea8-4baf-9bc3-01082d9dc9bf None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:01.758987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-b3588972-cea8-4baf-9bc3-01082d9dc9bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:00:01.758987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:01.758987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:01.758987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:01.758987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:01.758987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:01.758987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:01.758987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:01.758987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:01.758987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:01.758987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:01.758987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:01.758987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:01.758987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:01.758987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:01.758987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:01.759984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:01.760947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:01.760947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:01.760947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:01.760947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:01.760947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:01.760947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:01.760947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:01.760947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:01.760947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:00:01.760947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:01.760947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 281/561] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jul 20 09:00:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:01.761567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3588972-cea8-4baf-9bc3-01082d9dc9bf None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:01.782537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:01.788980 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e746e-cce5-46d4-8f37-690aa8162cf4 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:01.827585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653eedc-08e6-46c7-a61d-809b0dd7eb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:01.828433 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653eedc-08e6-46c7-a61d-809b0dd7eb04 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:01.828714 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653eedc-08e6-46c7-a61d-809b0dd7eb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:01.829199 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653eedc-08e6-46c7-a61d-809b0dd7eb04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:01.836195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653eedc-08e6-46c7-a61d-809b0dd7eb04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:01.838833 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653eedc-08e6-46c7-a61d-809b0dd7eb04 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:01.853396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-6653eedc-08e6-46c7-a61d-809b0dd7eb04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:01.853396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:01.853396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:01.853396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:01.853396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:01.853396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:01.853396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:01.853396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:01.853396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:01.853396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:01.853396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:01.853396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:01.853396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:01.853396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:01.853396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:01.854316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:01.855250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:01.855250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:01.855250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:01.855250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:01.855250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:01.855250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:01.855250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:01.855250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:01.859057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 281/562] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:01.860285 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653eedc-08e6-46c7-a61d-809b0dd7eb04 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:01.879819 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:01.883860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3588972-cea8-4baf-9bc3-01082d9dc9bf None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:01.926288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fccbc8-313a-41b5-9c95-0a20f006373f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:01.927166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fccbc8-313a-41b5-9c95-0a20f006373f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:01.927496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fccbc8-313a-41b5-9c95-0a20f006373f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:01.927787 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fccbc8-313a-41b5-9c95-0a20f006373f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:01.940858 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fccbc8-313a-41b5-9c95-0a20f006373f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:01.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-69fccbc8-313a-41b5-9c95-0a20f006373f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:01.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:01.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:01.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:01.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:01.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:01.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:01.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:01.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:01.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:01.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:01.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:01.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:01.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:01.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:01.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:01.953927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:01.953927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:01.953927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:01.953927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:01.953927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:01.953927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:01.953927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:01.953927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:01.953927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:01.953927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:01.953927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:01.953927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:01.953927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:01.953927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:01.953927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:01.953927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:01.960140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:01.960140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:01.960140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:01.960140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:01.960140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:01.960140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:01.960140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:01.960140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:01.960140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:01.960140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:01.960140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:01.960140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:01.960140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 282/563] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:00:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:01.960140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c71ae-23fa-4f70-81ae-c3c2aad0bf90 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:01.960913 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c71ae-23fa-4f70-81ae-c3c2aad0bf90 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:01.960913 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c71ae-23fa-4f70-81ae-c3c2aad0bf90 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:01.960913 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 283/564] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:01.960913 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c71ae-23fa-4f70-81ae-c3c2aad0bf90 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:01.967117 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653eedc-08e6-46c7-a61d-809b0dd7eb04 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:02.001634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b740c4-08f1-4dd9-99b2-05e922800004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:02.002300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b740c4-08f1-4dd9-99b2-05e922800004 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:02.002583 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b740c4-08f1-4dd9-99b2-05e922800004 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:02.002891 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b740c4-08f1-4dd9-99b2-05e922800004 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:02.003723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b740c4-08f1-4dd9-99b2-05e922800004 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:02.004031 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b740c4-08f1-4dd9-99b2-05e922800004 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:02.005468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b740c4-08f1-4dd9-99b2-05e922800004 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:02.022210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 282/565] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jul 20 09:00:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:02.023037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b740c4-08f1-4dd9-99b2-05e922800004 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:02.041982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:02.067578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c71ae-23fa-4f70-81ae-c3c2aad0bf90 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:02.100925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:02.101134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440b079-48b1-4b6c-a729-6073e56a1733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:02.101800 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440b079-48b1-4b6c-a729-6073e56a1733 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:02.102145 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440b079-48b1-4b6c-a729-6073e56a1733 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:02.102381 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440b079-48b1-4b6c-a729-6073e56a1733 None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:02.103314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440b079-48b1-4b6c-a729-6073e56a1733 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:02.103595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440b079-48b1-4b6c-a729-6073e56a1733 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:02.105038 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440b079-48b1-4b6c-a729-6073e56a1733 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:02.117503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 284/566] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:02.118737 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58633db3-dce7-4a17-aa8a-e6fc17f56552 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:02.119038 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58633db3-dce7-4a17-aa8a-e6fc17f56552 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:02.119387 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58633db3-dce7-4a17-aa8a-e6fc17f56552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:02.126257 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b740c4-08f1-4dd9-99b2-05e922800004 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:02.135907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-58633db3-dce7-4a17-aa8a-e6fc17f56552 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:02.137101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-58633db3-dce7-4a17-aa8a-e6fc17f56552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:02.159694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ebb83-0a96-4100-ad33-98ec8d102b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:02.160359 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ebb83-0a96-4100-ad33-98ec8d102b75 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:02.160639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ebb83-0a96-4100-ad33-98ec8d102b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:02.160908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ebb83-0a96-4100-ad33-98ec8d102b75 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:02.161645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ebb83-0a96-4100-ad33-98ec8d102b75 None admin] RBAC: Adding query filter params (name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:02.161948 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ebb83-0a96-4100-ad33-98ec8d102b75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:02.163332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ebb83-0a96-4100-ad33-98ec8d102b75 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:02.175943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 283/567] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 09:00:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:02.175943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90de18-23a7-4223-ac86-79cfe311ab08 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:02.175943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90de18-23a7-4223-ac86-79cfe311ab08 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:02.175943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90de18-23a7-4223-ac86-79cfe311ab08 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:02.175943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 284/568] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:02.175943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90de18-23a7-4223-ac86-79cfe311ab08 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:02.207925 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 285/569] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:02.208764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58633db3-dce7-4a17-aa8a-e6fc17f56552 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:02.273595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:02.288038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90de18-23a7-4223-ac86-79cfe311ab08 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:02.309957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58633db3-dce7-4a17-aa8a-e6fc17f56552 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:02.330420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5901f2-c965-4edd-b59c-66195114685b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:02.331698 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5901f2-c965-4edd-b59c-66195114685b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:02.332164 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5901f2-c965-4edd-b59c-66195114685b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:02.332645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5901f2-c965-4edd-b59c-66195114685b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:02.344707 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:02.350439 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5901f2-c965-4edd-b59c-66195114685b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:02.352802 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5901f2-c965-4edd-b59c-66195114685b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:02.358057 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18abdc-be13-44e3-9af5-d3743806bfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:02.359113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18abdc-be13-44e3-9af5-d3743806bfae None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:02.359575 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18abdc-be13-44e3-9af5-d3743806bfae None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:02.360032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18abdc-be13-44e3-9af5-d3743806bfae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:02.361501 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18abdc-be13-44e3-9af5-d3743806bfae None admin] RBAC: Adding query filter params (role.id=ae949d8bea4b40d9b1a9fa207d539663, scope.project.id=a498284274454d25b1b118a88996e522, user.id=03b4375e6ead42fb8d77a7b30617cb95) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:02.362018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18abdc-be13-44e3-9af5-d3743806bfae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:02.364393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18abdc-be13-44e3-9af5-d3743806bfae None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:02.373055 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 286/570] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:00:02 2022] GET /identity/v3/role_assignments?user.id=03b4375e6ead42fb8d77a7b30617cb95&scope.project.id=a498284274454d25b1b118a88996e522&role.id=ae949d8bea4b40d9b1a9fa207d539663 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:02.374114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18abdc-be13-44e3-9af5-d3743806bfae None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:02.380289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-6b5901f2-c965-4edd-b59c-66195114685b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:02.380289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:02.380289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:02.380289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:02.380289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:02.380289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:02.380289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:02.380289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:02.380289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:02.380289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:02.380289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:02.380289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:02.380289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:02.380289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:02.380289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:02.381225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:02.382303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:02.382303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:02.382303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:02.382303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:02.382303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 285/571] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:02.383157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc3aa5-b4b2-444d-9aaa-f1207587a257 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:02.383445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc3aa5-b4b2-444d-9aaa-f1207587a257 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:02.383722 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc3aa5-b4b2-444d-9aaa-f1207587a257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:02.400790 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-7cbc3aa5-b4b2-444d-9aaa-f1207587a257 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:02.402029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-7cbc3aa5-b4b2-444d-9aaa-f1207587a257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:02.475138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 286/572] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:02.476115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc3aa5-b4b2-444d-9aaa-f1207587a257 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:02.512417 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18abdc-be13-44e3-9af5-d3743806bfae None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:02.552850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:02.558569 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ea78c-7db7-4eb2-b9cb-e7236166151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:02.559475 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ea78c-7db7-4eb2-b9cb-e7236166151d None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:02.559766 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ea78c-7db7-4eb2-b9cb-e7236166151d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:02.560048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ea78c-7db7-4eb2-b9cb-e7236166151d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:02.571689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ea78c-7db7-4eb2-b9cb-e7236166151d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:02.573140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ea78c-7db7-4eb2-b9cb-e7236166151d None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:02.591289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-6b5ea78c-7db7-4eb2-b9cb-e7236166151d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:02.591289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:02.591289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:02.591289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:02.591289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:02.591289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:02.591289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:02.591289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:02.591289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:02.591289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:02.591289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:02.591289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:02.591289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:02.591289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:02.591289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:02.592301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:02.593174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:02.593174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:02.593174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:02.593174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:02.593174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 287/573] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:02.593856 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ea78c-7db7-4eb2-b9cb-e7236166151d None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:02.610375 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc3aa5-b4b2-444d-9aaa-f1207587a257 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:02.634708 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:02.652740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745a3e2-0778-415f-be4d-0e6683e5ce85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:02.653537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745a3e2-0778-415f-be4d-0e6683e5ce85 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:02.653819 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745a3e2-0778-415f-be4d-0e6683e5ce85 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:02.654157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745a3e2-0778-415f-be4d-0e6683e5ce85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:02.665004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745a3e2-0778-415f-be4d-0e6683e5ce85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:02.673227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-8745a3e2-0778-415f-be4d-0e6683e5ce85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:02.673227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:02.673227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:02.673227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:02.673227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:02.673227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:02.673227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:02.673227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:02.673227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:02.673227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:02.673227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:02.673227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:02.673227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:02.673227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:02.673227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:02.673227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:02.674167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:02.674167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:02.674167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:02.674167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:02.674167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:02.674167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:02.674167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:02.674167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:02.674167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:02.674167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:02.674167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:02.674167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:02.674167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:02.674167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:02.674167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:02.674167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:02.675033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:02.675033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:02.675033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:02.675033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:02.675033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:02.675033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:02.675033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:02.675033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:02.675033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:02.675033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:02.675033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:02.675033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:02.675033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 287/574] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:00:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:02.675720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745a3e2-0778-415f-be4d-0e6683e5ce85 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:02.712178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ea78c-7db7-4eb2-b9cb-e7236166151d None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:02.749499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2898130-2b12-4342-b756-71a269ba5d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:02.750157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2898130-2b12-4342-b756-71a269ba5d3e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:02.750446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2898130-2b12-4342-b756-71a269ba5d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:02.750779 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2898130-2b12-4342-b756-71a269ba5d3e None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:02.751510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2898130-2b12-4342-b756-71a269ba5d3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:02.751823 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2898130-2b12-4342-b756-71a269ba5d3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:02.753184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2898130-2b12-4342-b756-71a269ba5d3e None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:02.764887 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 288/575] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:02.765711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2898130-2b12-4342-b756-71a269ba5d3e None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:02.789012 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745a3e2-0778-415f-be4d-0e6683e5ce85 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:02.815412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:02.828182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fcf48-c710-4aa2-b508-0bdd1bf02ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:02.829305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fcf48-c710-4aa2-b508-0bdd1bf02ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:02.829592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fcf48-c710-4aa2-b508-0bdd1bf02ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:02.829872 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fcf48-c710-4aa2-b508-0bdd1bf02ab4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:02.840004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3fcf48-c710-4aa2-b508-0bdd1bf02ab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:02.847669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-ec3fcf48-c710-4aa2-b508-0bdd1bf02ab4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:02.847669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:02.847669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:02.847669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:02.847669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:02.847669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:02.847669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:02.847669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:02.847669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:02.847669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:02.847669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:02.847669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:02.847669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:02.847669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:02.847669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:02.847669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:02.848777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:02.848777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:02.848777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:02.848777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:02.848777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:02.848777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:02.848777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:02.848777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:02.848777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:02.848777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:02.848777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:02.848777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:02.848777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:02.848777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:02.848777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:02.848777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:02.849637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:02.849637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:02.849637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:02.849637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:02.849637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:02.849637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:02.849637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:02.849637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:02.849637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:02.849637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:02.849637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:02.849637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:02.849637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 288/576] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:00:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:02.849637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fcf48-c710-4aa2-b508-0bdd1bf02ab4 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:02.879803 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2898130-2b12-4342-b756-71a269ba5d3e None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:02.893688 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:02.917310 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0600a-ad2a-494d-b5b7-06083e08775d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:02.917971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0600a-ad2a-494d-b5b7-06083e08775d None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:02.918258 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0600a-ad2a-494d-b5b7-06083e08775d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:02.918544 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0600a-ad2a-494d-b5b7-06083e08775d None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:02.919299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0600a-ad2a-494d-b5b7-06083e08775d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:02.919616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0600a-ad2a-494d-b5b7-06083e08775d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:02.921638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0600a-ad2a-494d-b5b7-06083e08775d None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:02.934148 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 289/577] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:02.934987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0600a-ad2a-494d-b5b7-06083e08775d None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:02.959116 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fcf48-c710-4aa2-b508-0bdd1bf02ab4 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:02.996609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd95682-7861-461a-902f-f2082337ff91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:02.997562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd95682-7861-461a-902f-f2082337ff91 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:02.998062 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd95682-7861-461a-902f-f2082337ff91 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:02.998697 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd95682-7861-461a-902f-f2082337ff91 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:02.999622 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd95682-7861-461a-902f-f2082337ff91 None admin] RBAC: Adding query filter params (name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:03.000073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd95682-7861-461a-902f-f2082337ff91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:03.001588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd95682-7861-461a-902f-f2082337ff91 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:03.009728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 289/578] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 09:00:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:03.010719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd95682-7861-461a-902f-f2082337ff91 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:03.050860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0600a-ad2a-494d-b5b7-06083e08775d None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:03.053237 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:03.090692 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa81f6-9299-47c3-8b94-63943242e322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:03.091501 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa81f6-9299-47c3-8b94-63943242e322 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:03.091785 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa81f6-9299-47c3-8b94-63943242e322 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:03.092060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa81f6-9299-47c3-8b94-63943242e322 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:03.100314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa81f6-9299-47c3-8b94-63943242e322 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:03.101766 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa81f6-9299-47c3-8b94-63943242e322 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:03.116058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-80aa81f6-9299-47c3-8b94-63943242e322 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:03.116058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:03.116058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:03.116058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:03.116058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:03.116058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:03.116058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:03.116058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:03.116058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:03.116058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:03.116058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:03.116058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:03.116058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:03.116058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:03.116058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:03.116954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:03.117825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:03.117825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:03.117825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:03.117825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:03.117825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:03.117825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:03.117825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:03.117825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:03.117825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 290/579] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:03.118364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa81f6-9299-47c3-8b94-63943242e322 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:03.125350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd95682-7861-461a-902f-f2082337ff91 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:03.127051 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:03.162580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79699b21-4fe4-47b9-9b1d-fe8634107adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:03.163196 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79699b21-4fe4-47b9-9b1d-fe8634107adb None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:03.163478 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79699b21-4fe4-47b9-9b1d-fe8634107adb None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:03.163752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79699b21-4fe4-47b9-9b1d-fe8634107adb None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:03.164494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79699b21-4fe4-47b9-9b1d-fe8634107adb None admin] RBAC: Adding query filter params (name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:03.164796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79699b21-4fe4-47b9-9b1d-fe8634107adb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:03.166173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79699b21-4fe4-47b9-9b1d-fe8634107adb None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:03.174459 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 290/580] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 09:00:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:03.175557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79699b21-4fe4-47b9-9b1d-fe8634107adb None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:03.229742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa81f6-9299-47c3-8b94-63943242e322 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:03.266725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0520f64-8cbe-4be6-8cf5-94bc540dd04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:03.267524 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0520f64-8cbe-4be6-8cf5-94bc540dd04f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:03.267820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0520f64-8cbe-4be6-8cf5-94bc540dd04f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:03.268089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0520f64-8cbe-4be6-8cf5-94bc540dd04f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:03.275895 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0520f64-8cbe-4be6-8cf5-94bc540dd04f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:03.277328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0520f64-8cbe-4be6-8cf5-94bc540dd04f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:03.287471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79699b21-4fe4-47b9-9b1d-fe8634107adb None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:03.292415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-e0520f64-8cbe-4be6-8cf5-94bc540dd04f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 09:00:03.292415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:03.292415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:03.292415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:03.292415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:03.292415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:03.292415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:03.292415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:03.292415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:03.292415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:03.292415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:03.292415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:03.292415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:03.292415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:03.292415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:03.292415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:03.293392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:03.294350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:03.294350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:03.294350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:03.294350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:03.294350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:03.294350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:03.294350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:03.294350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:03.294350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 09:00:03.294350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:03.298749 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 291/581] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:00:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:03.299496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0520f64-8cbe-4be6-8cf5-94bc540dd04f None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:03.324802 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3797f-c85b-4cd2-909b-349a29291f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:03.325596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3797f-c85b-4cd2-909b-349a29291f42 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:03.325878 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3797f-c85b-4cd2-909b-349a29291f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:03.326771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3797f-c85b-4cd2-909b-349a29291f42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:03.328183 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:03.338852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3797f-c85b-4cd2-909b-349a29291f42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:03.340302 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3797f-c85b-4cd2-909b-349a29291f42 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:03.359222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-49b3797f-c85b-4cd2-909b-349a29291f42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:03.359222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:03.359222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:03.359222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:03.359222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:03.359222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:03.359222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:03.359222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:03.359222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:03.359222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:03.359222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:03.359222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:03.359222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:03.359222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:03.359222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:03.360160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:03.361082 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:03.361082 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:03.361082 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:03.361082 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:03.361082 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 291/582] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:03.361365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3797f-c85b-4cd2-909b-349a29291f42 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:03.381528 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:03.417628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0520f64-8cbe-4be6-8cf5-94bc540dd04f None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:03.457013 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3ca89-9dc7-4b3d-b9ad-54c5a7e69ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:03.457687 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3ca89-9dc7-4b3d-b9ad-54c5a7e69ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:03.457976 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3ca89-9dc7-4b3d-b9ad-54c5a7e69ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:03.458266 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3ca89-9dc7-4b3d-b9ad-54c5a7e69ef3 None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:03.459090 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3ca89-9dc7-4b3d-b9ad-54c5a7e69ef3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:03.459409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3ca89-9dc7-4b3d-b9ad-54c5a7e69ef3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:03.460836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3ca89-9dc7-4b3d-b9ad-54c5a7e69ef3 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:03.472070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3797f-c85b-4cd2-909b-349a29291f42 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:03.479749 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 292/583] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:03.480568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3ca89-9dc7-4b3d-b9ad-54c5a7e69ef3 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:03.511435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946b952-b687-472e-9f09-848f8355716e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:03.512228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946b952-b687-472e-9f09-848f8355716e None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:03.512509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946b952-b687-472e-9f09-848f8355716e None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:03.512779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946b952-b687-472e-9f09-848f8355716e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:03.524153 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946b952-b687-472e-9f09-848f8355716e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:03.525602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946b952-b687-472e-9f09-848f8355716e None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:03.543194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-a946b952-b687-472e-9f09-848f8355716e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:03.543194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:03.543194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:03.543194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:03.543194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:03.543194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:03.543194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:03.543194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:03.543194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:03.543194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:03.543194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:03.543194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:03.543194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:03.543194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:03.543194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:03.544112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:03.545107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:03.545107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:03.545107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:03.545107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:03.545107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 292/584] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:03.545954 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946b952-b687-472e-9f09-848f8355716e None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:03.579493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:03.599749 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3ca89-9dc7-4b3d-b9ad-54c5a7e69ef3 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:03.619150 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:03.638367 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785d46c-499b-4a95-8ad7-b3dd84e1e362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:03.640058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785d46c-499b-4a95-8ad7-b3dd84e1e362 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:03.640401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785d46c-499b-4a95-8ad7-b3dd84e1e362 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:03.640689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785d46c-499b-4a95-8ad7-b3dd84e1e362 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:03.641592 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785d46c-499b-4a95-8ad7-b3dd84e1e362 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:03.641910 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785d46c-499b-4a95-8ad7-b3dd84e1e362 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:03.644216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785d46c-499b-4a95-8ad7-b3dd84e1e362 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:03.655399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946b952-b687-472e-9f09-848f8355716e None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:03.663148 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 293/585] 10.222.0.44 () {56 vars in 1137 bytes} [Wed Jul 20 09:00:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:03.663781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785d46c-499b-4a95-8ad7-b3dd84e1e362 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:03.696931 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5840d-3b5f-4c2d-bd00-6bc8650913e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:03.697523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5840d-3b5f-4c2d-bd00-6bc8650913e7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:03.697808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5840d-3b5f-4c2d-bd00-6bc8650913e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:03.698085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5840d-3b5f-4c2d-bd00-6bc8650913e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:03.698864 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5840d-3b5f-4c2d-bd00-6bc8650913e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:03.699170 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5840d-3b5f-4c2d-bd00-6bc8650913e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:03.700539 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5840d-3b5f-4c2d-bd00-6bc8650913e7 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:03.718882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 293/586] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:03.718882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5840d-3b5f-4c2d-bd00-6bc8650913e7 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:03.775715 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785d46c-499b-4a95-8ad7-b3dd84e1e362 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:03.812238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85173ca-3fe8-4b9c-b57d-863250a32075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:03.813122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85173ca-3fe8-4b9c-b57d-863250a32075 None admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:03.813417 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85173ca-3fe8-4b9c-b57d-863250a32075 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:03.813672 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85173ca-3fe8-4b9c-b57d-863250a32075 None admin] PATH_INFO: `/v3/projects/a498284274454d25b1b118a88996e522/users/be08c43f0d0c4a8194596e54500b44ed/roles/ae949d8bea4b40d9b1a9fa207d539663` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:03.824445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5840d-3b5f-4c2d-bd00-6bc8650913e7 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:03.826418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:03.831753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85173ca-3fe8-4b9c-b57d-863250a32075 None admin] RBAC: Authorizing `identity:create_grant(project_id=a498284274454d25b1b118a88996e522, user_id=be08c43f0d0c4a8194596e54500b44ed, role_id=ae949d8bea4b40d9b1a9fa207d539663)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:03.834825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85173ca-3fe8-4b9c-b57d-863250a32075 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:03.849636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 294/587] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 09:00:03 2022] PUT /identity/v3/projects/a498284274454d25b1b118a88996e522/users/be08c43f0d0c4a8194596e54500b44ed/roles/ae949d8bea4b40d9b1a9fa207d539663 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:00:03.852582 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:03.860057 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85173ca-3fe8-4b9c-b57d-863250a32075 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:03.861209 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696110c-f18a-41db-925a-919da299012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:03.861889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696110c-f18a-41db-925a-919da299012c None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:03.862815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696110c-f18a-41db-925a-919da299012c None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:03.862815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696110c-f18a-41db-925a-919da299012c None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:03.863211 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696110c-f18a-41db-925a-919da299012c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:03.863499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696110c-f18a-41db-925a-919da299012c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:03.864957 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696110c-f18a-41db-925a-919da299012c None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:03.880167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 294/588] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:03.881096 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696110c-f18a-41db-925a-919da299012c None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:04.002824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85173ca-3fe8-4b9c-b57d-863250a32075 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:04.011378 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696110c-f18a-41db-925a-919da299012c None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:04.049486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651814f-932f-4a4b-a979-75ec161277ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:04.050483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651814f-932f-4a4b-a979-75ec161277ec None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:04.050815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651814f-932f-4a4b-a979-75ec161277ec None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:04.051107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651814f-932f-4a4b-a979-75ec161277ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:04.056865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d482e-b1b5-482f-93e3-a56346c3f5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:04.057669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d482e-b1b5-482f-93e3-a56346c3f5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:04.057948 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d482e-b1b5-482f-93e3-a56346c3f5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:04.058223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d482e-b1b5-482f-93e3-a56346c3f5e2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:04.067695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651814f-932f-4a4b-a979-75ec161277ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:04.068759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d482e-b1b5-482f-93e3-a56346c3f5e2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:04.068873 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651814f-932f-4a4b-a979-75ec161277ec None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:04.070085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d482e-b1b5-482f-93e3-a56346c3f5e2 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:04.086961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-da5d482e-b1b5-482f-93e3-a56346c3f5e2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 09:00:04.086961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:04.086961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:04.086961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:04.086961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:04.086961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:04.086961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:04.086961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:04.086961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:04.086961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:04.086961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:04.086961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:04.086961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:04.086961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:04.086961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:04.086961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:04.087924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:04.088848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:04.088848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:04.088848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:04.088848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:04.088848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:04.088848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:04.088848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:04.088848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:04.088848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 09:00:04.088848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:04.088848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 295/589] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:00:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:04.089508 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d482e-b1b5-482f-93e3-a56346c3f5e2 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:04.101820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-5651814f-932f-4a4b-a979-75ec161277ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:04.101820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:04.101820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:04.101820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:04.101820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:04.101820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:04.101820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:04.101820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:04.101820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:04.101820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:04.101820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:04.101820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:04.101820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:04.101820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:04.101820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:04.102966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:04.103992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:04.103992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:04.103992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:04.103992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:04.105967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 295/590] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:04.111700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651814f-932f-4a4b-a979-75ec161277ec None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:04.152453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:04.159329 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:04.209668 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d482e-b1b5-482f-93e3-a56346c3f5e2 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:04.235535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651814f-932f-4a4b-a979-75ec161277ec None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:04.246331 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb26a15-e384-436d-9daa-c09f092dbf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:04.247470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb26a15-e384-436d-9daa-c09f092dbf9f None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:04.247999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb26a15-e384-436d-9daa-c09f092dbf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:04.248509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb26a15-e384-436d-9daa-c09f092dbf9f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:04.257362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb26a15-e384-436d-9daa-c09f092dbf9f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:04.259140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb26a15-e384-436d-9daa-c09f092dbf9f None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:04.275694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-ddb26a15-e384-436d-9daa-c09f092dbf9f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 09:00:04.275694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:04.275694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:04.275694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:04.275694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:04.275694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:04.275694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:04.275694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:04.275694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:04.275694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:04.275694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:04.275694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:04.275694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:04.275694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:04.275694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:04.275694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:04.276674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:04.277592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:04.277592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:04.277592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:04.277592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:04.277592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:04.277592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:04.277592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:04.277592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:04.277592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 09:00:04.277592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:04.278111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6b8a0-4d54-403a-8b39-2cb15a47f69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:04.278111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6b8a0-4d54-403a-8b39-2cb15a47f69e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:04.278111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6b8a0-4d54-403a-8b39-2cb15a47f69e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:04.278111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6b8a0-4d54-403a-8b39-2cb15a47f69e None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:04.278965 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6b8a0-4d54-403a-8b39-2cb15a47f69e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:04.279109 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6b8a0-4d54-403a-8b39-2cb15a47f69e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:04.281224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6b8a0-4d54-403a-8b39-2cb15a47f69e None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:04.281649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 296/591] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:00:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:04.282727 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb26a15-e384-436d-9daa-c09f092dbf9f None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:04.310058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 296/592] 10.222.0.44 () {56 vars in 1137 bytes} [Wed Jul 20 09:00:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:04.310058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6b8a0-4d54-403a-8b39-2cb15a47f69e None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:04.405124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb26a15-e384-436d-9daa-c09f092dbf9f None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:04.414737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:04.426885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6b8a0-4d54-403a-8b39-2cb15a47f69e None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:04.428396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:04.445469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e52de-00e7-4ef2-abf7-259ef2e2da31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:04.446132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e52de-00e7-4ef2-abf7-259ef2e2da31 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:04.446446 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e52de-00e7-4ef2-abf7-259ef2e2da31 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:04.446698 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e52de-00e7-4ef2-abf7-259ef2e2da31 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:04.447510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e52de-00e7-4ef2-abf7-259ef2e2da31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:04.447818 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e52de-00e7-4ef2-abf7-259ef2e2da31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:04.449192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e52de-00e7-4ef2-abf7-259ef2e2da31 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:04.467870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 297/593] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:04.467870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e52de-00e7-4ef2-abf7-259ef2e2da31 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:04.469135 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2245c2-d86a-47d3-a5b7-b2e656a41ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:04.469819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2245c2-d86a-47d3-a5b7-b2e656a41ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:04.470104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2245c2-d86a-47d3-a5b7-b2e656a41ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:04.470382 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2245c2-d86a-47d3-a5b7-b2e656a41ac6 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:04.471263 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2245c2-d86a-47d3-a5b7-b2e656a41ac6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:04.472363 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2245c2-d86a-47d3-a5b7-b2e656a41ac6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:04.473012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2245c2-d86a-47d3-a5b7-b2e656a41ac6 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:04.496487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 297/594] 10.222.0.44 () {56 vars in 1139 bytes} [Wed Jul 20 09:00:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:04.496487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17673de6-d2de-4b60-87d9-2cce20f06417 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:04.496487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17673de6-d2de-4b60-87d9-2cce20f06417 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:04.496487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17673de6-d2de-4b60-87d9-2cce20f06417 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:04.496487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 298/595] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:04.496487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17673de6-d2de-4b60-87d9-2cce20f06417 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:04.589285 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e52de-00e7-4ef2-abf7-259ef2e2da31 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:04.608869 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17673de6-d2de-4b60-87d9-2cce20f06417 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:04.627787 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40afff-c168-4d6a-a5b6-0fcc5c312e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:04.628589 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40afff-c168-4d6a-a5b6-0fcc5c312e92 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:04.628870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40afff-c168-4d6a-a5b6-0fcc5c312e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:04.629147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40afff-c168-4d6a-a5b6-0fcc5c312e92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:04.639996 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40afff-c168-4d6a-a5b6-0fcc5c312e92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:04.641451 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40afff-c168-4d6a-a5b6-0fcc5c312e92 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:04.644880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb2864-8f08-45cf-b62f-5e7e70b89ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:04.645674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb2864-8f08-45cf-b62f-5e7e70b89ace None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:04.645962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb2864-8f08-45cf-b62f-5e7e70b89ace None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:04.646267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb2864-8f08-45cf-b62f-5e7e70b89ace None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:04.653501 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb2864-8f08-45cf-b62f-5e7e70b89ace None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:04.655637 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb2864-8f08-45cf-b62f-5e7e70b89ace None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:04.659729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-ec40afff-c168-4d6a-a5b6-0fcc5c312e92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:04.659729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:04.659729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:04.659729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:04.659729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:04.659729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:04.659729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:04.659729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:04.659729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:04.659729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:04.659729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:04.659729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:04.659729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:04.659729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:04.659729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:04.660612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:04.661523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:04.661523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:04.661523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:04.661523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:04.661523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 298/596] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:04.661804 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40afff-c168-4d6a-a5b6-0fcc5c312e92 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:04.675139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-87fb2864-8f08-45cf-b62f-5e7e70b89ace None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:04.675139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:04.675139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:04.675139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:04.675139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:04.675139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:04.675139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:04.675139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:04.675139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:04.675139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:04.675139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:04.675139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:04.675139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:04.675139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:04.675139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:04.676246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:04.677169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:04.677169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:04.677169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:04.677169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:04.677169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:04.677169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:04.677169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:04.677169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:04.677169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 299/597] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:04.677650 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c0910-fb18-4903-bcd9-7b7684b5221e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:04.677815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c0910-fb18-4903-bcd9-7b7684b5221e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:04.678101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c0910-fb18-4903-bcd9-7b7684b5221e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:04.689932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:04.696671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-a96c0910-fb18-4903-bcd9-7b7684b5221e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:04.697881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-a96c0910-fb18-4903-bcd9-7b7684b5221e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:04.712928 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:04.769113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 300/598] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:04.770094 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c0910-fb18-4903-bcd9-7b7684b5221e None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:04.790447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40afff-c168-4d6a-a5b6-0fcc5c312e92 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:04.824938 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235e6ec-2575-4a86-8da1-f75d1b9819b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:04.825744 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235e6ec-2575-4a86-8da1-f75d1b9819b5 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:04.826570 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235e6ec-2575-4a86-8da1-f75d1b9819b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:04.826874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235e6ec-2575-4a86-8da1-f75d1b9819b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:04.837288 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235e6ec-2575-4a86-8da1-f75d1b9819b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:04.838742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235e6ec-2575-4a86-8da1-f75d1b9819b5 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:04.855963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-3235e6ec-2575-4a86-8da1-f75d1b9819b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:04.855963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:04.855963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:04.855963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:04.855963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:04.855963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:04.855963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:04.855963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:04.855963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:04.855963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:04.855963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:04.855963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:04.855963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:04.855963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:04.855963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:04.856895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:04.857961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:04.857961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:04.857961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:04.857961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:04.857961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 299/599] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:04.857961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235e6ec-2575-4a86-8da1-f75d1b9819b5 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:04.885782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c0910-fb18-4903-bcd9-7b7684b5221e None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:04.921991 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56743bbe-911c-437a-bb9f-1035adaff7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:04.922657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56743bbe-911c-437a-bb9f-1035adaff7db None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:04.922968 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56743bbe-911c-437a-bb9f-1035adaff7db None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:04.923242 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56743bbe-911c-437a-bb9f-1035adaff7db None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:04.923963 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56743bbe-911c-437a-bb9f-1035adaff7db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:04.924267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56743bbe-911c-437a-bb9f-1035adaff7db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:04.925860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56743bbe-911c-437a-bb9f-1035adaff7db None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:04.929429 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:04.939184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 301/600] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:04.940012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56743bbe-911c-437a-bb9f-1035adaff7db None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:04.953768 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:04.966919 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235e6ec-2575-4a86-8da1-f75d1b9819b5 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:05.002437 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364c9b2-608b-4bf2-95ed-663061011e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:05.003290 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364c9b2-608b-4bf2-95ed-663061011e56 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:05.003688 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364c9b2-608b-4bf2-95ed-663061011e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:05.004070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364c9b2-608b-4bf2-95ed-663061011e56 None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:05.004876 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364c9b2-608b-4bf2-95ed-663061011e56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:05.005193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364c9b2-608b-4bf2-95ed-663061011e56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:05.006704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364c9b2-608b-4bf2-95ed-663061011e56 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:05.020002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 300/601] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:05.021161 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364c9b2-608b-4bf2-95ed-663061011e56 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:05.050550 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56743bbe-911c-437a-bb9f-1035adaff7db None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:05.092216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d589bf4-b488-4123-95ee-c20e0529b5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:05.092944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d589bf4-b488-4123-95ee-c20e0529b5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:05.093229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d589bf4-b488-4123-95ee-c20e0529b5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:05.093512 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d589bf4-b488-4123-95ee-c20e0529b5f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:05.104585 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d589bf4-b488-4123-95ee-c20e0529b5f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:05.113340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-7d589bf4-b488-4123-95ee-c20e0529b5f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:05.113340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:05.113340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:05.113340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:05.113340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:05.113340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:05.113340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:05.113340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:05.113340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:05.113340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:05.113340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:05.113340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:05.113340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:05.113340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:05.113340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:05.113340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:05.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:05.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:05.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:05.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:05.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:05.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:05.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:05.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:05.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:05.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:05.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:05.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:05.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:05.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:05.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:05.114295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:05.116035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:05.116035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:05.116035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:05.116035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:05.116035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:05.116035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:05.116035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:05.116035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:05.116035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:05.116035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:05.116035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:05.116035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:05.116035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 302/602] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:00:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:05.116035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d589bf4-b488-4123-95ee-c20e0529b5f6 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:05.143867 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364c9b2-608b-4bf2-95ed-663061011e56 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:05.175721 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:05.182015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa81a2-1a11-4ad2-aa17-b4ad80c3ec50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:05.182700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa81a2-1a11-4ad2-aa17-b4ad80c3ec50 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:05.182993 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa81a2-1a11-4ad2-aa17-b4ad80c3ec50 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:05.183276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa81a2-1a11-4ad2-aa17-b4ad80c3ec50 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:05.183991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa81a2-1a11-4ad2-aa17-b4ad80c3ec50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:05.184300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa81a2-1a11-4ad2-aa17-b4ad80c3ec50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:05.186190 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa81a2-1a11-4ad2-aa17-b4ad80c3ec50 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:05.192830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:05.204600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 301/603] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:05.204600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa81a2-1a11-4ad2-aa17-b4ad80c3ec50 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:05.231494 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d589bf4-b488-4123-95ee-c20e0529b5f6 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:05.267001 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b6c70-34cf-4b66-a292-a51b08451a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:05.267908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b6c70-34cf-4b66-a292-a51b08451a06 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:05.268264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b6c70-34cf-4b66-a292-a51b08451a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:05.268622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b6c70-34cf-4b66-a292-a51b08451a06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:05.275871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b6c70-34cf-4b66-a292-a51b08451a06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:05.277448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b6c70-34cf-4b66-a292-a51b08451a06 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:05.293661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-816b6c70-34cf-4b66-a292-a51b08451a06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:05.293661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:05.293661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:05.293661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:05.293661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:05.293661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:05.293661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:05.293661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:05.293661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:05.293661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:05.293661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:05.293661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:05.293661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:05.293661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:05.293661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:05.295478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:05.296537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:05.296537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:05.296537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:05.296537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:05.296537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:05.296537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:05.296537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:05.296537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:05.297028 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 303/604] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:05.297719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b6c70-34cf-4b66-a292-a51b08451a06 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:05.309892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa81a2-1a11-4ad2-aa17-b4ad80c3ec50 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:05.347762 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc2a02-f648-4e9f-b78e-b2909dca50d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:05.348567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc2a02-f648-4e9f-b78e-b2909dca50d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:05.348845 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc2a02-f648-4e9f-b78e-b2909dca50d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:05.349119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc2a02-f648-4e9f-b78e-b2909dca50d6 None admin] PATH_INFO: `/v3/projects/a498284274454d25b1b118a88996e522/users/bb93eb1d25c4410591125b0ab7a73149/roles/ae949d8bea4b40d9b1a9fa207d539663` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:05.365019 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc2a02-f648-4e9f-b78e-b2909dca50d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a498284274454d25b1b118a88996e522, user_id=bb93eb1d25c4410591125b0ab7a73149, role_id=ae949d8bea4b40d9b1a9fa207d539663)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:05.366456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc2a02-f648-4e9f-b78e-b2909dca50d6 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:05.386804 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 302/605] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 09:00:05 2022] PUT /identity/v3/projects/a498284274454d25b1b118a88996e522/users/bb93eb1d25c4410591125b0ab7a73149/roles/ae949d8bea4b40d9b1a9fa207d539663 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:00:05.387917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc2a02-f648-4e9f-b78e-b2909dca50d6 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:05.417131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b6c70-34cf-4b66-a292-a51b08451a06 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:05.459169 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:05.465435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:05.473960 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69324cf2-cf46-4465-b204-966e8dc6ade3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:05.475165 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69324cf2-cf46-4465-b204-966e8dc6ade3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:05.475547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69324cf2-cf46-4465-b204-966e8dc6ade3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:05.476195 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69324cf2-cf46-4465-b204-966e8dc6ade3 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:05.477079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69324cf2-cf46-4465-b204-966e8dc6ade3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:05.478015 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69324cf2-cf46-4465-b204-966e8dc6ade3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:05.479388 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69324cf2-cf46-4465-b204-966e8dc6ade3 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:05.490345 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 304/606] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 09:00:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:05.491367 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69324cf2-cf46-4465-b204-966e8dc6ade3 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:05.536630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc2a02-f648-4e9f-b78e-b2909dca50d6 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:05.578986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb382961-e7dc-4205-a340-56b0e3ae6153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:05.580089 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb382961-e7dc-4205-a340-56b0e3ae6153 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:05.580511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb382961-e7dc-4205-a340-56b0e3ae6153 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:05.580950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb382961-e7dc-4205-a340-56b0e3ae6153 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:05.588703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb382961-e7dc-4205-a340-56b0e3ae6153 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:05.590346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb382961-e7dc-4205-a340-56b0e3ae6153 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:05.606246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-fb382961-e7dc-4205-a340-56b0e3ae6153 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:05.606246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:05.606246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:05.606246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:05.606246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:05.606246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:05.606246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:05.606246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:05.606246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:05.606246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:05.606246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:05.606246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:05.606246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:05.606246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:05.606246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:05.607413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:05.608479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:05.608479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:05.608479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:05.608479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:05.608479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:05.608479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:05.608479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:05.608479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:05.613601 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69324cf2-cf46-4465-b204-966e8dc6ade3 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:05.614761 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 303/607] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:05.615625 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb382961-e7dc-4205-a340-56b0e3ae6153 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:05.649270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222ea63-440e-47f8-8505-1fc01fe09f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:05.650186 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222ea63-440e-47f8-8505-1fc01fe09f5e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:05.650571 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222ea63-440e-47f8-8505-1fc01fe09f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:05.650974 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222ea63-440e-47f8-8505-1fc01fe09f5e None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:05.651888 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222ea63-440e-47f8-8505-1fc01fe09f5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:05.652302 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222ea63-440e-47f8-8505-1fc01fe09f5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:05.654830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222ea63-440e-47f8-8505-1fc01fe09f5e None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:05.668258 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 305/608] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:05.673862 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222ea63-440e-47f8-8505-1fc01fe09f5e None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:05.725811 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:05.735937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb382961-e7dc-4205-a340-56b0e3ae6153 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:05.737867 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:05.775510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede6bd0-4d1c-4662-96ab-624c307c1f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:05.776544 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede6bd0-4d1c-4662-96ab-624c307c1f87 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:05.776969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede6bd0-4d1c-4662-96ab-624c307c1f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:05.777382 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede6bd0-4d1c-4662-96ab-624c307c1f87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:05.778453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222ea63-440e-47f8-8505-1fc01fe09f5e None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:05.788600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede6bd0-4d1c-4662-96ab-624c307c1f87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:05.790364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede6bd0-4d1c-4662-96ab-624c307c1f87 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:05.810131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-6ede6bd0-4d1c-4662-96ab-624c307c1f87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:05.810131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:05.810131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:05.810131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:05.810131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:05.810131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:05.810131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:05.810131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:05.810131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:05.810131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:05.810131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:05.810131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:05.810131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:05.810131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:05.810131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:05.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:05.812470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:05.812470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:05.812470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:05.812470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:05.814322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 304/609] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:05.815321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede6bd0-4d1c-4662-96ab-624c307c1f87 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:05.821429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa3e6f-6ada-4c50-aebe-cd256f42f328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:05.822169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa3e6f-6ada-4c50-aebe-cd256f42f328 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:05.822705 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa3e6f-6ada-4c50-aebe-cd256f42f328 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:05.823548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa3e6f-6ada-4c50-aebe-cd256f42f328 None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:05.823949 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa3e6f-6ada-4c50-aebe-cd256f42f328 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:05.824377 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa3e6f-6ada-4c50-aebe-cd256f42f328 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:05.826559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa3e6f-6ada-4c50-aebe-cd256f42f328 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:05.846138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 306/610] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:05.846138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa3e6f-6ada-4c50-aebe-cd256f42f328 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:05.955926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede6bd0-4d1c-4662-96ab-624c307c1f87 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:05.961446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa3e6f-6ada-4c50-aebe-cd256f42f328 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:05.975991 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:06.002527 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f291309-f90c-4888-8d9d-9daeb4eb522b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:06.003558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f291309-f90c-4888-8d9d-9daeb4eb522b None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:06.003953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f291309-f90c-4888-8d9d-9daeb4eb522b None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:06.004375 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f291309-f90c-4888-8d9d-9daeb4eb522b None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:06.004863 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d6576-4272-4d8e-b2ec-bb506713e592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:06.005297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f291309-f90c-4888-8d9d-9daeb4eb522b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:06.005753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f291309-f90c-4888-8d9d-9daeb4eb522b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:06.006205 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d6576-4272-4d8e-b2ec-bb506713e592 None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:06.006626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d6576-4272-4d8e-b2ec-bb506713e592 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:06.007066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d6576-4272-4d8e-b2ec-bb506713e592 None admin] PATH_INFO: `/v3/projects/a498284274454d25b1b118a88996e522/users/99dc3db93e9045b8b3ef59a030822972/roles/ae949d8bea4b40d9b1a9fa207d539663` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:06.007547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f291309-f90c-4888-8d9d-9daeb4eb522b None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:06.020985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 307/611] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:06.026334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f291309-f90c-4888-8d9d-9daeb4eb522b None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:06.027069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d6576-4272-4d8e-b2ec-bb506713e592 None admin] RBAC: Authorizing `identity:create_grant(project_id=a498284274454d25b1b118a88996e522, user_id=99dc3db93e9045b8b3ef59a030822972, role_id=ae949d8bea4b40d9b1a9fa207d539663)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:06.028679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d6576-4272-4d8e-b2ec-bb506713e592 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:06.054772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 305/612] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 09:00:05 2022] PUT /identity/v3/projects/a498284274454d25b1b118a88996e522/users/99dc3db93e9045b8b3ef59a030822972/roles/ae949d8bea4b40d9b1a9fa207d539663 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:00:06.054772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d6576-4272-4d8e-b2ec-bb506713e592 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:06.063330 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:06.188054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f291309-f90c-4888-8d9d-9daeb4eb522b None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:06.190192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d6576-4272-4d8e-b2ec-bb506713e592 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:06.231236 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6ff43-6ea4-4fac-9e9d-7e5c20b58944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:06.231956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6ff43-6ea4-4fac-9e9d-7e5c20b58944 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:06.232322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6ff43-6ea4-4fac-9e9d-7e5c20b58944 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:06.232560 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6ff43-6ea4-4fac-9e9d-7e5c20b58944 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:06.234240 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2c151-9cfd-497c-9615-8dffe0fac7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:06.235172 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2c151-9cfd-497c-9615-8dffe0fac7ac None admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:06.235546 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2c151-9cfd-497c-9615-8dffe0fac7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:06.235723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2c151-9cfd-497c-9615-8dffe0fac7ac None admin] PATH_INFO: `/v3/projects/a498284274454d25b1b118a88996e522/users/5e5ad4e122ae471cad4cf1b0e3111dd9/roles/ae949d8bea4b40d9b1a9fa207d539663` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:06.242661 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6ff43-6ea4-4fac-9e9d-7e5c20b58944 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:06.244151 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6ff43-6ea4-4fac-9e9d-7e5c20b58944 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:06.255559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2c151-9cfd-497c-9615-8dffe0fac7ac None admin] RBAC: Authorizing `identity:create_grant(project_id=a498284274454d25b1b118a88996e522, user_id=5e5ad4e122ae471cad4cf1b0e3111dd9, role_id=ae949d8bea4b40d9b1a9fa207d539663)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:06.257214 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2c151-9cfd-497c-9615-8dffe0fac7ac None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:06.261579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-08c6ff43-6ea4-4fac-9e9d-7e5c20b58944 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:00:06.261579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:06.261579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:06.261579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:06.261579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:06.261579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:06.261579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:06.261579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:06.261579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:06.261579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:06.261579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:06.261579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:06.261579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:06.261579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:06.261579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:06.261579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:06.262630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:06.263855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:06.263855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:06.263855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:06.263855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:06.263855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:06.263855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:06.263855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:06.263855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:06.263855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:00:06.263855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:06.263855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 306/613] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:00:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:06.265351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ced22-3c6c-4a0e-b1fa-c7764049f124 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:06.265751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ced22-3c6c-4a0e-b1fa-c7764049f124 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:06.265853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ced22-3c6c-4a0e-b1fa-c7764049f124 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:06.267337 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 307/614] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:06.272255 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ced22-3c6c-4a0e-b1fa-c7764049f124 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:06.274880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 308/615] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 09:00:06 2022] PUT /identity/v3/projects/a498284274454d25b1b118a88996e522/users/5e5ad4e122ae471cad4cf1b0e3111dd9/roles/ae949d8bea4b40d9b1a9fa207d539663 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:00:06.277449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf0f7d-bcfd-4e6b-9f68-db972aac9796 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:06.277741 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf0f7d-bcfd-4e6b-9f68-db972aac9796 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:06.278018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf0f7d-bcfd-4e6b-9f68-db972aac9796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:06.300184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-f6cf0f7d-bcfd-4e6b-9f68-db972aac9796 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:06.301417 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f6cf0f7d-bcfd-4e6b-9f68-db972aac9796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:06.347124 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:06.393881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:06.433507 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 309/616] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:06.441526 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ced22-3c6c-4a0e-b1fa-c7764049f124 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:06.464629 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf0f7d-bcfd-4e6b-9f68-db972aac9796 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:06.485561 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f1592-1bcf-4910-9f55-50d00b435598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:06.486466 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1592-1bcf-4910-9f55-50d00b435598 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:06.486806 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1592-1bcf-4910-9f55-50d00b435598 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:06.487095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f1592-1bcf-4910-9f55-50d00b435598 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:06.487946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f1592-1bcf-4910-9f55-50d00b435598 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:06.488316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f1592-1bcf-4910-9f55-50d00b435598 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:06.490060 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f1592-1bcf-4910-9f55-50d00b435598 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:06.515510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 308/617] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Jul 20 09:00:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:06.521119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f1592-1bcf-4910-9f55-50d00b435598 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:06.592542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf0f7d-bcfd-4e6b-9f68-db972aac9796 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:06.628776 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618be1f-7345-47be-a92b-0b556374b4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:06.629881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618be1f-7345-47be-a92b-0b556374b4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:06.629991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f1592-1bcf-4910-9f55-50d00b435598 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:06.630456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618be1f-7345-47be-a92b-0b556374b4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:06.630961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618be1f-7345-47be-a92b-0b556374b4a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:06.643128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7618be1f-7345-47be-a92b-0b556374b4a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:06.651919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-7618be1f-7345-47be-a92b-0b556374b4a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:06.651919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:06.651919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:06.651919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:06.651919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:06.651919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:06.651919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:06.651919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:06.651919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:06.651919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:06.651919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:06.651919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:06.651919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:06.651919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:06.651919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:06.651919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:06.652847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:06.652847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:06.652847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:06.652847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:06.652847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:06.652847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:06.652847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:06.652847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:06.652847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:06.652847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:06.652847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:06.652847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:06.652847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:06.652847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:06.652847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:06.652847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:06.653688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:06.653688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:06.653688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:06.653688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:06.653688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:06.653688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:06.653688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:06.653688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:06.653688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:06.653688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:06.653688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:06.653688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:06.655837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:06.657613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 310/618] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:00:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:06.664151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618be1f-7345-47be-a92b-0b556374b4a2 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:06.675390 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4493c5-382c-434b-82fa-6bdab6e0afe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:06.676247 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4493c5-382c-434b-82fa-6bdab6e0afe2 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:06.676529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4493c5-382c-434b-82fa-6bdab6e0afe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:06.676807 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4493c5-382c-434b-82fa-6bdab6e0afe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:06.684964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:06.693708 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4493c5-382c-434b-82fa-6bdab6e0afe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:06.695464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4493c5-382c-434b-82fa-6bdab6e0afe2 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:06.733351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-8e4493c5-382c-434b-82fa-6bdab6e0afe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:06.733351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:06.733351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:06.733351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:06.733351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:06.733351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:06.733351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:06.733351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:06.733351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:06.733351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:06.733351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:06.733351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:06.733351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:06.733351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:06.733351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:06.734314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:06.735371 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:06.735371 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:06.735371 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:06.735371 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:06.735371 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 309/619] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:06.750479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4493c5-382c-434b-82fa-6bdab6e0afe2 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:06.876605 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618be1f-7345-47be-a92b-0b556374b4a2 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:06.883425 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4493c5-382c-434b-82fa-6bdab6e0afe2 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:06.925462 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cada41e-5ff5-469b-ba6c-88a91456ef31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:06.926135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cada41e-5ff5-469b-ba6c-88a91456ef31 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:06.926485 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cada41e-5ff5-469b-ba6c-88a91456ef31 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:06.926706 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cada41e-5ff5-469b-ba6c-88a91456ef31 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:06.927447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cada41e-5ff5-469b-ba6c-88a91456ef31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:06.927757 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cada41e-5ff5-469b-ba6c-88a91456ef31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:06.929156 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cada41e-5ff5-469b-ba6c-88a91456ef31 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:06.951901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf4651-6476-4077-9a88-dd48a8da1cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:06.952577 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf4651-6476-4077-9a88-dd48a8da1cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:06.952859 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf4651-6476-4077-9a88-dd48a8da1cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:06.953135 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf4651-6476-4077-9a88-dd48a8da1cc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:06.953891 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf4651-6476-4077-9a88-dd48a8da1cc2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:06.954228 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf4651-6476-4077-9a88-dd48a8da1cc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:06.955598 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf4651-6476-4077-9a88-dd48a8da1cc2 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:06.957280 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 310/620] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:06.958184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cada41e-5ff5-469b-ba6c-88a91456ef31 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:06.977150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 311/621] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 09:00:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:07.004217 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf4651-6476-4077-9a88-dd48a8da1cc2 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:07.020537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:07.095090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cada41e-5ff5-469b-ba6c-88a91456ef31 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:07.135400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb11531-a200-40e0-9214-72bcd9a96a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:07.136200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb11531-a200-40e0-9214-72bcd9a96a12 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:07.136486 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb11531-a200-40e0-9214-72bcd9a96a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:07.136767 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb11531-a200-40e0-9214-72bcd9a96a12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:07.155078 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb11531-a200-40e0-9214-72bcd9a96a12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:07.156707 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb11531-a200-40e0-9214-72bcd9a96a12 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:07.178512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-bcb11531-a200-40e0-9214-72bcd9a96a12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:07.178512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:07.178512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:07.178512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:07.178512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:07.178512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:07.178512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:07.178512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:07.178512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:07.178512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:07.178512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:07.178512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:07.178512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:07.178512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:07.178512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:07.179495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:07.180388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:07.180388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:07.180388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:07.180388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:07.180388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:07.180388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:07.180388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:07.180388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:07.184942 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 311/622] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:07.196017 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb11531-a200-40e0-9214-72bcd9a96a12 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:07.207748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:07.258964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf4651-6476-4077-9a88-dd48a8da1cc2 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:07.327500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:07.328768 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db7867-bade-4caf-b0b4-db11e92199b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:07.329798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db7867-bade-4caf-b0b4-db11e92199b5 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:07.330279 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db7867-bade-4caf-b0b4-db11e92199b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:07.331217 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db7867-bade-4caf-b0b4-db11e92199b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:07.345815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb11531-a200-40e0-9214-72bcd9a96a12 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:07.353711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db7867-bade-4caf-b0b4-db11e92199b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:07.359023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db7867-bade-4caf-b0b4-db11e92199b5 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:07.392901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-06db7867-bade-4caf-b0b4-db11e92199b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:07.392901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:07.392901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:07.392901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:07.392901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:07.392901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:07.392901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:07.392901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:07.392901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:07.392901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:07.392901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:07.392901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:07.392901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:07.392901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:07.392901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:07.393884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:07.394798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:07.394798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:07.394798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:07.394798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:07.398776 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 312/623] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:07.400068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db7867-bade-4caf-b0b4-db11e92199b5 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:07.414197 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775e826-0d9e-4d80-b8ed-15b8091bf898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:07.415265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775e826-0d9e-4d80-b8ed-15b8091bf898 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:07.415845 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775e826-0d9e-4d80-b8ed-15b8091bf898 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:07.416394 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775e826-0d9e-4d80-b8ed-15b8091bf898 None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:07.417380 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775e826-0d9e-4d80-b8ed-15b8091bf898 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:07.417889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775e826-0d9e-4d80-b8ed-15b8091bf898 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:07.419778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775e826-0d9e-4d80-b8ed-15b8091bf898 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:07.442465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 312/624] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:07.453636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775e826-0d9e-4d80-b8ed-15b8091bf898 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:07.554995 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db7867-bade-4caf-b0b4-db11e92199b5 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:07.584357 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:07.594715 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f1575-8d59-4970-91fc-c2ff182a7e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:07.595407 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f1575-8d59-4970-91fc-c2ff182a7e92 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:07.595690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f1575-8d59-4970-91fc-c2ff182a7e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:07.595972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f1575-8d59-4970-91fc-c2ff182a7e92 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:07.596708 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f1575-8d59-4970-91fc-c2ff182a7e92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:07.597017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f1575-8d59-4970-91fc-c2ff182a7e92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:07.598399 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f1575-8d59-4970-91fc-c2ff182a7e92 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:07.611508 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 313/625] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:07.612895 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bac248-6c55-42d9-94bc-bac6f76c33b8 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:07.613118 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bac248-6c55-42d9-94bc-bac6f76c33b8 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:07.613383 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bac248-6c55-42d9-94bc-bac6f76c33b8 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:07.618771 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 314/626] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:07.618771 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bac248-6c55-42d9-94bc-bac6f76c33b8 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:07.637104 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775e826-0d9e-4d80-b8ed-15b8091bf898 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:07.675571 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878fed7-d589-493b-a6be-838122754049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:07.676459 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878fed7-d589-493b-a6be-838122754049 None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:07.676737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878fed7-d589-493b-a6be-838122754049 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:07.677010 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878fed7-d589-493b-a6be-838122754049 None admin] PATH_INFO: `/v3/projects/a498284274454d25b1b118a88996e522/users/03b4375e6ead42fb8d77a7b30617cb95/roles/ae949d8bea4b40d9b1a9fa207d539663` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:07.693881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878fed7-d589-493b-a6be-838122754049 None admin] RBAC: Authorizing `identity:create_grant(project_id=a498284274454d25b1b118a88996e522, user_id=03b4375e6ead42fb8d77a7b30617cb95, role_id=ae949d8bea4b40d9b1a9fa207d539663)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:07.695743 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878fed7-d589-493b-a6be-838122754049 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:07.718711 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 313/627] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 09:00:07 2022] PUT /identity/v3/projects/a498284274454d25b1b118a88996e522/users/03b4375e6ead42fb8d77a7b30617cb95/roles/ae949d8bea4b40d9b1a9fa207d539663 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:00:07.718711 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64ab5b-3732-4b7c-9271-882b1da03639 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:07.718711 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64ab5b-3732-4b7c-9271-882b1da03639 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:07.718711 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be64ab5b-3732-4b7c-9271-882b1da03639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:07.729685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:07.732264 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-be64ab5b-3732-4b7c-9271-882b1da03639 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:07.733523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-be64ab5b-3732-4b7c-9271-882b1da03639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:07.788813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bac248-6c55-42d9-94bc-bac6f76c33b8 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:07.864288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070ad52-da3f-477b-9814-d82475488fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:07.865096 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070ad52-da3f-477b-9814-d82475488fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:07.865379 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070ad52-da3f-477b-9814-d82475488fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:07.865658 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070ad52-da3f-477b-9814-d82475488fa6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:07.866491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 314/628] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:07.882430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070ad52-da3f-477b-9814-d82475488fa6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:07.884101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6070ad52-da3f-477b-9814-d82475488fa6 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:07.889887 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64ab5b-3732-4b7c-9271-882b1da03639 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:07.918541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-6070ad52-da3f-477b-9814-d82475488fa6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:00:07.918541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:07.918541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:07.918541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:07.918541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:07.918541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:07.918541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:07.918541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:07.918541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:07.918541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:07.918541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:07.918541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:07.918541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:07.918541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:07.918541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:07.918541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:07.920330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:07.921446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:07.921446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:07.921446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:07.921446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:07.921446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:07.921446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:07.921446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:07.921446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:07.921446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:00:07.921446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:07.924280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 315/629] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jul 20 09:00:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:07.929663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070ad52-da3f-477b-9814-d82475488fa6 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:07.988947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:08.004783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be64ab5b-3732-4b7c-9271-882b1da03639 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:08.055566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677cb4b-d701-459d-a24c-a94669c23789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:08.056388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677cb4b-d701-459d-a24c-a94669c23789 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:08.056707 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677cb4b-d701-459d-a24c-a94669c23789 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:08.056950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677cb4b-d701-459d-a24c-a94669c23789 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:08.059215 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:08.060455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6070ad52-da3f-477b-9814-d82475488fa6 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:08.080666 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677cb4b-d701-459d-a24c-a94669c23789 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:08.093865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-9677cb4b-d701-459d-a24c-a94669c23789 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:08.093865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:08.093865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:08.093865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:08.093865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:08.093865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:08.093865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:08.093865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:08.093865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:08.093865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:08.093865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:08.093865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:08.093865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:08.093865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:08.093865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:08.093865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:08.094877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:08.094877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:08.094877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:08.094877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:08.094877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:08.094877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:08.094877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:08.094877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:08.094877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:08.094877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:08.094877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:08.094877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:08.094877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:08.094877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:08.094877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:08.094877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:08.095734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:08.095734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:08.095734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:08.095734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:08.095734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:08.095734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:08.095734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:08.095734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:08.095734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:08.095734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:08.095734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:08.095734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:08.095734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 315/630] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:00:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:08.102476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677cb4b-d701-459d-a24c-a94669c23789 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:08.112301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482418d0-7a92-4689-8343-816eaffa9b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:08.112977 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482418d0-7a92-4689-8343-816eaffa9b30 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:08.113257 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482418d0-7a92-4689-8343-816eaffa9b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:08.113535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482418d0-7a92-4689-8343-816eaffa9b30 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:08.114426 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482418d0-7a92-4689-8343-816eaffa9b30 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:08.114721 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482418d0-7a92-4689-8343-816eaffa9b30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:08.116302 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482418d0-7a92-4689-8343-816eaffa9b30 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:08.134908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 316/631] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jul 20 09:00:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:08.141429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482418d0-7a92-4689-8343-816eaffa9b30 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:08.309538 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677cb4b-d701-459d-a24c-a94669c23789 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:08.346482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482418d0-7a92-4689-8343-816eaffa9b30 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:08.352247 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f312a75-ca76-417d-891f-27454ea5850d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:08.352898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:08.354078 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f312a75-ca76-417d-891f-27454ea5850d None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:08.354574 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f312a75-ca76-417d-891f-27454ea5850d None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:08.355114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f312a75-ca76-417d-891f-27454ea5850d None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:08.356069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f312a75-ca76-417d-891f-27454ea5850d None admin] RBAC: Adding query filter params (name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:08.356578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f312a75-ca76-417d-891f-27454ea5850d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:08.358250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f312a75-ca76-417d-891f-27454ea5850d None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:08.367545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 316/632] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 09:00:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:08.373534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f312a75-ca76-417d-891f-27454ea5850d None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:08.416829 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bae9d3-91ff-44fd-b104-aa82fa3b9726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:08.417493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bae9d3-91ff-44fd-b104-aa82fa3b9726 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:08.417778 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bae9d3-91ff-44fd-b104-aa82fa3b9726 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:08.418059 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bae9d3-91ff-44fd-b104-aa82fa3b9726 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:08.475132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:08.502432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bae9d3-91ff-44fd-b104-aa82fa3b9726 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:08.503949 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bae9d3-91ff-44fd-b104-aa82fa3b9726 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:08.529729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f312a75-ca76-417d-891f-27454ea5850d None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:08.591256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c5ea9-a48b-49c5-993c-3243bd43d5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:08.592526 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c5ea9-a48b-49c5-993c-3243bd43d5cd None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:08.593015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c5ea9-a48b-49c5-993c-3243bd43d5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:08.593505 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c5ea9-a48b-49c5-993c-3243bd43d5cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:08.605073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c5ea9-a48b-49c5-993c-3243bd43d5cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:08.606857 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c5ea9-a48b-49c5-993c-3243bd43d5cd None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:08.613825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 317/633] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:00:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:00:08.615349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1943a7-4c60-4346-ad14-b2dcb58408e8 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:08.615646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1943a7-4c60-4346-ad14-b2dcb58408e8 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:08.615933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1943a7-4c60-4346-ad14-b2dcb58408e8 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:08.617886 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 318/634] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:08.618586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1943a7-4c60-4346-ad14-b2dcb58408e8 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:08.623021 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:08.628297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-a50c5ea9-a48b-49c5-993c-3243bd43d5cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:08.628297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:08.628297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:08.628297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:08.628297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:08.628297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:08.628297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:08.628297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:08.628297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:08.628297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:08.628297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:08.628297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:08.628297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:08.628297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:08.628297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:08.629183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:08.630073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:08.630073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:08.630073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:08.630073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:08.632450 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 317/635] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:08.634013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d5b2d-09a3-4115-a10c-275c33be0472 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:08.634500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d5b2d-09a3-4115-a10c-275c33be0472 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:08.635049 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d5b2d-09a3-4115-a10c-275c33be0472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:08.653190 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-331d5b2d-09a3-4115-a10c-275c33be0472 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:08.655682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-331d5b2d-09a3-4115-a10c-275c33be0472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:08.728077 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 318/636] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:08.729516 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d5b2d-09a3-4115-a10c-275c33be0472 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:08.739356 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1943a7-4c60-4346-ad14-b2dcb58408e8 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:08.772276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:08.777198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a24ab-e70d-4f23-989e-adf241ebba7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:08.778050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a24ab-e70d-4f23-989e-adf241ebba7e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:08.778233 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a24ab-e70d-4f23-989e-adf241ebba7e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:08.778484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a24ab-e70d-4f23-989e-adf241ebba7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:08.792253 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a24ab-e70d-4f23-989e-adf241ebba7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:08.793705 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a24ab-e70d-4f23-989e-adf241ebba7e None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:08.812535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-339a24ab-e70d-4f23-989e-adf241ebba7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:08.812535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:08.812535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:08.812535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:08.812535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:08.812535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:08.812535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:08.812535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:08.812535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:08.812535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:08.812535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:08.812535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:08.812535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:08.812535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:08.812535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:08.813564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:08.814505 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:08.814505 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:08.814505 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:08.814505 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:08.818339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 319/637] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:08.819599 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a24ab-e70d-4f23-989e-adf241ebba7e None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:08.842286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d5b2d-09a3-4115-a10c-275c33be0472 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:08.877656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959cc1b0-9d09-4be6-b19d-5718f259d1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:08.878652 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cc1b0-9d09-4be6-b19d-5718f259d1b4 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:08.879198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cc1b0-9d09-4be6-b19d-5718f259d1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:08.879689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cc1b0-9d09-4be6-b19d-5718f259d1b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:08.880617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959cc1b0-9d09-4be6-b19d-5718f259d1b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:08.881189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959cc1b0-9d09-4be6-b19d-5718f259d1b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:08.882828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959cc1b0-9d09-4be6-b19d-5718f259d1b4 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:08.894333 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:08.895861 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 319/638] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:08.896774 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959cc1b0-9d09-4be6-b19d-5718f259d1b4 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:08.937409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a24ab-e70d-4f23-989e-adf241ebba7e None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:08.975281 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b303be-e9d3-40bb-abe6-12dd8ebc426e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:08.976088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b303be-e9d3-40bb-abe6-12dd8ebc426e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:08.976370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b303be-e9d3-40bb-abe6-12dd8ebc426e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:08.977048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b303be-e9d3-40bb-abe6-12dd8ebc426e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:08.988643 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b303be-e9d3-40bb-abe6-12dd8ebc426e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:08.996783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-71b303be-e9d3-40bb-abe6-12dd8ebc426e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:08.996783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:08.996783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:08.996783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:08.996783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:08.996783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:08.996783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:08.996783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:08.996783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:08.996783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:08.996783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:08.996783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:08.996783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:08.996783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:08.996783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:08.996783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:08.998238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:08.998238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:08.998238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:08.998238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:08.998238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:08.998238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:08.998238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:08.998238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:08.998238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:08.998238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:08.998238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:08.998238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:08.998238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:08.998238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:08.998238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:08.998238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:08.999772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:08.999772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:08.999772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:08.999772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:08.999772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:08.999772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:08.999772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:08.999772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:08.999772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:08.999772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:08.999772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:08.999772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:08.999772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 320/639] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:00:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:08.999772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a0710-f1e7-4882-8f30-1e9ca670d099 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:09.000565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a0710-f1e7-4882-8f30-1e9ca670d099 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:09.000565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a0710-f1e7-4882-8f30-1e9ca670d099 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:09.001737 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 321/640] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:09.002190 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a0710-f1e7-4882-8f30-1e9ca670d099 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:09.008430 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:09.012161 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959cc1b0-9d09-4be6-b19d-5718f259d1b4 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:09.049974 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c743c94-528d-4620-824d-c27e1485d180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:09.050695 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c743c94-528d-4620-824d-c27e1485d180 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:09.050995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c743c94-528d-4620-824d-c27e1485d180 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:09.051275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c743c94-528d-4620-824d-c27e1485d180 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:09.052003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c743c94-528d-4620-824d-c27e1485d180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:09.052310 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c743c94-528d-4620-824d-c27e1485d180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:09.053695 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c743c94-528d-4620-824d-c27e1485d180 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:09.068674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 320/641] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:09.074419 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c743c94-528d-4620-824d-c27e1485d180 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:09.119550 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a0710-f1e7-4882-8f30-1e9ca670d099 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:09.136294 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:09.155637 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82375ad1-2166-47e3-9967-f7fd83a8942e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:09.156448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82375ad1-2166-47e3-9967-f7fd83a8942e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:09.156731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82375ad1-2166-47e3-9967-f7fd83a8942e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:09.157009 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82375ad1-2166-47e3-9967-f7fd83a8942e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:09.164920 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82375ad1-2166-47e3-9967-f7fd83a8942e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:09.166353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82375ad1-2166-47e3-9967-f7fd83a8942e None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:09.180798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-82375ad1-2166-47e3-9967-f7fd83a8942e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 09:00:09.180798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:09.180798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:09.180798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:09.180798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:09.180798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:09.180798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:09.180798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:09.180798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:09.180798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:09.180798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:09.180798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:09.180798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:09.180798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:09.180798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:09.180798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:09.181897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:09.183453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:09.183453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:09.183453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:09.183453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:09.183453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:09.183453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:09.183453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:09.183453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:09.183453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 09:00:09.183453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:09.183453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 322/642] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:00:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:09.184112 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864032b9-9d50-4512-b98b-b3483788a29c None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:09.184112 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864032b9-9d50-4512-b98b-b3483788a29c None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:09.184416 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864032b9-9d50-4512-b98b-b3483788a29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:09.199556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c743c94-528d-4620-824d-c27e1485d180 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:09.202309 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-864032b9-9d50-4512-b98b-b3483788a29c None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:09.203644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-864032b9-9d50-4512-b98b-b3483788a29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:09.237229 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d33c4a-d63e-42ab-806c-2932258c89ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:09.237892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d33c4a-d63e-42ab-806c-2932258c89ee None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:09.238176 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d33c4a-d63e-42ab-806c-2932258c89ee None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:09.238452 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d33c4a-d63e-42ab-806c-2932258c89ee None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:09.239277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d33c4a-d63e-42ab-806c-2932258c89ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:09.239590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d33c4a-d63e-42ab-806c-2932258c89ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:09.241428 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d33c4a-d63e-42ab-806c-2932258c89ee None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:09.249657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 321/643] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 09:00:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:09.254965 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d33c4a-d63e-42ab-806c-2932258c89ee None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:09.277373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 323/644] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:09.278575 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:09.278712 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864032b9-9d50-4512-b98b-b3483788a29c None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:09.370361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d33c4a-d63e-42ab-806c-2932258c89ee None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:09.394836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864032b9-9d50-4512-b98b-b3483788a29c None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:09.402534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:09.405720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f446b78-6b0f-46a0-9867-63070d675ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:09.406530 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f446b78-6b0f-46a0-9867-63070d675ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:09.406849 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f446b78-6b0f-46a0-9867-63070d675ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:09.407125 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f446b78-6b0f-46a0-9867-63070d675ca9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:09.414545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f446b78-6b0f-46a0-9867-63070d675ca9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:09.416053 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f446b78-6b0f-46a0-9867-63070d675ca9 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:09.431137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-0f446b78-6b0f-46a0-9867-63070d675ca9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:09.431137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:09.431137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:09.431137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:09.431137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:09.431137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:09.431137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:09.431137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:09.431137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:09.431137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:09.431137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:09.431137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:09.431137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:09.431137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:09.431137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:09.432114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:09.433195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:09.433195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:09.433195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:09.433195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:09.433195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:09.433195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:09.433195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:09.433195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:09.433644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97482a4-f595-42a9-8b39-f56b33940fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:09.433644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97482a4-f595-42a9-8b39-f56b33940fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:09.433644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97482a4-f595-42a9-8b39-f56b33940fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:09.433644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97482a4-f595-42a9-8b39-f56b33940fc9 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:09.433930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97482a4-f595-42a9-8b39-f56b33940fc9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:09.433930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97482a4-f595-42a9-8b39-f56b33940fc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:09.435769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97482a4-f595-42a9-8b39-f56b33940fc9 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:09.437805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 322/645] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:09.438572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f446b78-6b0f-46a0-9867-63070d675ca9 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:09.455100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 324/646] 10.222.0.44 () {56 vars in 1137 bytes} [Wed Jul 20 09:00:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:09.456188 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97482a4-f595-42a9-8b39-f56b33940fc9 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:09.527613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:09.551974 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f446b78-6b0f-46a0-9867-63070d675ca9 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:09.571461 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97482a4-f595-42a9-8b39-f56b33940fc9 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:09.587883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41151a-064c-4cf2-9947-9ef201fdcf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:09.589112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41151a-064c-4cf2-9947-9ef201fdcf28 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:09.589456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41151a-064c-4cf2-9947-9ef201fdcf28 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:09.589720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41151a-064c-4cf2-9947-9ef201fdcf28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:09.600759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41151a-064c-4cf2-9947-9ef201fdcf28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:09.602316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41151a-064c-4cf2-9947-9ef201fdcf28 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:09.611114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57790ee3-0d1f-41b6-905c-99f08134affe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:09.611921 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57790ee3-0d1f-41b6-905c-99f08134affe None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:09.612208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57790ee3-0d1f-41b6-905c-99f08134affe None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:09.612486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57790ee3-0d1f-41b6-905c-99f08134affe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:09.620862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-8b41151a-064c-4cf2-9947-9ef201fdcf28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:09.620862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:09.620862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:09.620862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:09.620862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:09.620862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:09.620862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:09.620862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:09.620862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:09.620862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:09.620862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:09.620862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:09.620862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:09.620862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:09.620862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:09.621786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:09.622757 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:09.622757 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:09.622757 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:09.622757 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:09.624262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57790ee3-0d1f-41b6-905c-99f08134affe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:09.626657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 323/647] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:09.627508 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41151a-064c-4cf2-9947-9ef201fdcf28 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:09.632270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-57790ee3-0d1f-41b6-905c-99f08134affe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:09.632270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:09.632270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:09.632270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:09.632270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:09.632270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:09.632270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:09.632270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:09.632270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:09.632270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:09.632270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:09.632270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:09.632270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:09.632270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:09.632270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:09.632270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:09.633220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:09.633220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:09.633220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:09.633220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:09.633220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:09.633220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:09.633220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:09.633220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:09.633220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:09.633220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:09.633220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:09.633220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:09.633220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:09.633220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:09.633220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:09.633220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:09.634072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:09.634072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:09.634072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:09.634072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:09.634072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:09.634072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:09.634072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:09.634072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:09.634072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:09.634072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:09.634072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:09.634072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:09.634072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 325/648] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:00:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:09.635963 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57790ee3-0d1f-41b6-905c-99f08134affe None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:09.662619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:09.735924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41151a-064c-4cf2-9947-9ef201fdcf28 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:09.746267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57790ee3-0d1f-41b6-905c-99f08134affe None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:09.766586 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:09.773189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1f6b2-4276-410c-b572-f08606199b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:09.773859 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1f6b2-4276-410c-b572-f08606199b77 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:09.774132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1f6b2-4276-410c-b572-f08606199b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:09.774416 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1f6b2-4276-410c-b572-f08606199b77 None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:09.775298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1f6b2-4276-410c-b572-f08606199b77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:09.775613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1f6b2-4276-410c-b572-f08606199b77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:09.777054 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e1f6b2-4276-410c-b572-f08606199b77 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:09.782307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017f12b-47f4-45cf-b797-c59be9968464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:09.782987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017f12b-47f4-45cf-b797-c59be9968464 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:09.783275 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017f12b-47f4-45cf-b797-c59be9968464 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:09.783555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017f12b-47f4-45cf-b797-c59be9968464 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:09.794953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 324/649] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:09.794953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1f6b2-4276-410c-b572-f08606199b77 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:09.857639 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017f12b-47f4-45cf-b797-c59be9968464 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:09.859327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017f12b-47f4-45cf-b797-c59be9968464 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:09.885529 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:09.894828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1f6b2-4276-410c-b572-f08606199b77 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:09.932849 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75c0a2-d356-4538-be6e-671197bbf843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:09.933436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75c0a2-d356-4538-be6e-671197bbf843 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:09.933725 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75c0a2-d356-4538-be6e-671197bbf843 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:09.934000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75c0a2-d356-4538-be6e-671197bbf843 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:09.934745 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75c0a2-d356-4538-be6e-671197bbf843 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:09.935053 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75c0a2-d356-4538-be6e-671197bbf843 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:09.937028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75c0a2-d356-4538-be6e-671197bbf843 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:09.953175 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 325/650] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:09.953175 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75c0a2-d356-4538-be6e-671197bbf843 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:09.977332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 326/651] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:00:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:00:09.977332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017f12b-47f4-45cf-b797-c59be9968464 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:10.006105 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:10.054734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75c0a2-d356-4538-be6e-671197bbf843 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:10.079422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017f12b-47f4-45cf-b797-c59be9968464 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:10.088557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da27a3-cac9-406a-bcd4-78fea73f3049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:10.089229 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da27a3-cac9-406a-bcd4-78fea73f3049 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:10.089513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da27a3-cac9-406a-bcd4-78fea73f3049 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:10.089794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da27a3-cac9-406a-bcd4-78fea73f3049 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:10.090541 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da27a3-cac9-406a-bcd4-78fea73f3049 None admin] RBAC: Adding query filter params (name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:10.090884 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da27a3-cac9-406a-bcd4-78fea73f3049 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:10.092292 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da27a3-cac9-406a-bcd4-78fea73f3049 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:10.100292 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 326/652] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 09:00:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:10.101086 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da27a3-cac9-406a-bcd4-78fea73f3049 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:10.103959 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:10.115750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1dad2-1dd8-4a33-9019-086ef3814d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:10.116411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1dad2-1dd8-4a33-9019-086ef3814d06 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:10.116693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1dad2-1dd8-4a33-9019-086ef3814d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:10.116969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e1dad2-1dd8-4a33-9019-086ef3814d06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:10.117799 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e1dad2-1dd8-4a33-9019-086ef3814d06 None admin] RBAC: Adding query filter params (role.id=ae949d8bea4b40d9b1a9fa207d539663, scope.project.id=a498284274454d25b1b118a88996e522, user.id=be08c43f0d0c4a8194596e54500b44ed) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:10.118117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e1dad2-1dd8-4a33-9019-086ef3814d06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:10.120987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e1dad2-1dd8-4a33-9019-086ef3814d06 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:10.128925 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 327/653] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:00:09 2022] GET /identity/v3/role_assignments?user.id=be08c43f0d0c4a8194596e54500b44ed&scope.project.id=a498284274454d25b1b118a88996e522&role.id=ae949d8bea4b40d9b1a9fa207d539663 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:10.129727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1dad2-1dd8-4a33-9019-086ef3814d06 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:10.215220 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da27a3-cac9-406a-bcd4-78fea73f3049 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:10.232486 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:10.248834 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e1dad2-1dd8-4a33-9019-086ef3814d06 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:10.263062 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3515d9-f265-4831-928b-905c193f455b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:10.265230 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3515d9-f265-4831-928b-905c193f455b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:10.265527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3515d9-f265-4831-928b-905c193f455b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:10.266225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3515d9-f265-4831-928b-905c193f455b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:10.277975 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3515d9-f265-4831-928b-905c193f455b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:10.279576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3515d9-f265-4831-928b-905c193f455b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:10.289002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49368e-8f26-4b95-a42e-5d7ce51a067a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:10.289002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49368e-8f26-4b95-a42e-5d7ce51a067a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:10.289298 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49368e-8f26-4b95-a42e-5d7ce51a067a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:10.289446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49368e-8f26-4b95-a42e-5d7ce51a067a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:10.295184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-7c3515d9-f265-4831-928b-905c193f455b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 09:00:10.295184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:10.295184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:10.295184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:10.295184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:10.295184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:10.295184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:10.295184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:10.295184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:10.295184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:10.295184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:10.295184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:10.295184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:10.295184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:10.295184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:10.295184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:10.296944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:10.298363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:10.298363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:10.298363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:10.298363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:10.298363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:10.298363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:10.298363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:10.298363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:10.298363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 09:00:10.298363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:10.298363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 327/654] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:00:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:10.299157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bd2bd-2ce1-40c4-8cca-a4e455bcdfff None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:10.299157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bd2bd-2ce1-40c4-8cca-a4e455bcdfff None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:10.299157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bd2bd-2ce1-40c4-8cca-a4e455bcdfff None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:10.304584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 328/655] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:10.305296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bd2bd-2ce1-40c4-8cca-a4e455bcdfff None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:10.306589 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49368e-8f26-4b95-a42e-5d7ce51a067a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:10.308091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49368e-8f26-4b95-a42e-5d7ce51a067a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:10.331476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-8d49368e-8f26-4b95-a42e-5d7ce51a067a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:10.331476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:10.331476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:10.331476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:10.331476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:10.331476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:10.331476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:10.331476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:10.331476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:10.331476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:10.331476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:10.331476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:10.331476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:10.331476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:10.331476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:10.332614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:10.333729 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:10.333729 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:10.333729 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:10.333729 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:10.335912 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 328/656] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:10.336905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49368e-8f26-4b95-a42e-5d7ce51a067a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:10.395517 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:10.439120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bd2bd-2ce1-40c4-8cca-a4e455bcdfff None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:10.465892 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49368e-8f26-4b95-a42e-5d7ce51a067a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:10.480001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca30ef5-c1e6-4abf-99be-4eebfe41f130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:10.481675 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca30ef5-c1e6-4abf-99be-4eebfe41f130 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:10.482557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca30ef5-c1e6-4abf-99be-4eebfe41f130 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:10.482893 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca30ef5-c1e6-4abf-99be-4eebfe41f130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:10.497411 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca30ef5-c1e6-4abf-99be-4eebfe41f130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:10.499591 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca30ef5-c1e6-4abf-99be-4eebfe41f130 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:10.507345 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfdfd4-1373-4079-b7fe-6b4005b94616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:10.507687 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfdfd4-1373-4079-b7fe-6b4005b94616 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:10.507913 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfdfd4-1373-4079-b7fe-6b4005b94616 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:10.508193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfdfd4-1373-4079-b7fe-6b4005b94616 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:10.509034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cfdfd4-1373-4079-b7fe-6b4005b94616 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:10.509344 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cfdfd4-1373-4079-b7fe-6b4005b94616 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:10.510834 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cfdfd4-1373-4079-b7fe-6b4005b94616 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:10.519537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-3ca30ef5-c1e6-4abf-99be-4eebfe41f130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:10.519537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:10.519537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:10.519537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:10.519537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:10.519537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:10.519537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:10.519537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:10.519537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:10.519537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:10.519537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:10.519537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:10.519537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:10.519537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:10.519537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:10.520464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:10.521407 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:10.521407 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:10.521407 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:10.521407 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:10.522990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 329/657] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:10.525849 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f5297-2565-493c-9a26-0903351343c2 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:10.526385 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f5297-2565-493c-9a26-0903351343c2 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:10.526965 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f5297-2565-493c-9a26-0903351343c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:10.534626 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 329/658] 10.222.0.44 () {56 vars in 1137 bytes} [Wed Jul 20 09:00:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:10.538175 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfdfd4-1373-4079-b7fe-6b4005b94616 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:10.548861 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-668f5297-2565-493c-9a26-0903351343c2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:10.550755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-668f5297-2565-493c-9a26-0903351343c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:10.575019 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:10.629690 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 330/659] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:10.630935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f5297-2565-493c-9a26-0903351343c2 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:10.670414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:10.675447 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfdfd4-1373-4079-b7fe-6b4005b94616 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:10.713232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69064a-c57a-48de-8129-100e7e9de9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:10.714175 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69064a-c57a-48de-8129-100e7e9de9cf None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:10.715360 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69064a-c57a-48de-8129-100e7e9de9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:10.715360 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69064a-c57a-48de-8129-100e7e9de9cf None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:10.715882 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69064a-c57a-48de-8129-100e7e9de9cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:10.716896 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69064a-c57a-48de-8129-100e7e9de9cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:10.718619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69064a-c57a-48de-8129-100e7e9de9cf None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:10.733907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 330/660] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:10.735169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69064a-c57a-48de-8129-100e7e9de9cf None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:10.749948 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668f5297-2565-493c-9a26-0903351343c2 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:10.792693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b658a4e-f87b-4d20-afba-201d728ca784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:10.793440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b658a4e-f87b-4d20-afba-201d728ca784 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:10.793721 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b658a4e-f87b-4d20-afba-201d728ca784 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:10.793992 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b658a4e-f87b-4d20-afba-201d728ca784 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:10.795376 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b658a4e-f87b-4d20-afba-201d728ca784 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:10.796289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b658a4e-f87b-4d20-afba-201d728ca784 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:10.798012 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b658a4e-f87b-4d20-afba-201d728ca784 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:10.813849 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 331/661] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:10.815108 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b658a4e-f87b-4d20-afba-201d728ca784 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:10.838137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:10.847084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69064a-c57a-48de-8129-100e7e9de9cf None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:10.885890 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067348cc-9116-4dca-99c0-a70306e9b6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:10.886706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067348cc-9116-4dca-99c0-a70306e9b6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:10.887010 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067348cc-9116-4dca-99c0-a70306e9b6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:10.887286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067348cc-9116-4dca-99c0-a70306e9b6f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:10.917343 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:10.940419 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b658a4e-f87b-4d20-afba-201d728ca784 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:10.966084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067348cc-9116-4dca-99c0-a70306e9b6f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:10.968972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067348cc-9116-4dca-99c0-a70306e9b6f2 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:10.985741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7e69c-e619-4536-bd21-d3911c78524a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:10.986705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7e69c-e619-4536-bd21-d3911c78524a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:10.987004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7e69c-e619-4536-bd21-d3911c78524a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:10.987376 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7e69c-e619-4536-bd21-d3911c78524a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:11.001733 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7e69c-e619-4536-bd21-d3911c78524a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:11.009840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-c4a7e69c-e619-4536-bd21-d3911c78524a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:11.009840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:11.009840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:11.009840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:11.009840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:11.009840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:11.009840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:11.009840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:11.009840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:11.009840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:11.009840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:11.009840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:11.009840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:11.009840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:11.009840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:11.009840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:11.010870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:11.010870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:11.010870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:11.010870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:11.010870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:11.010870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:11.010870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:11.010870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:11.010870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:11.010870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:11.010870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:11.010870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:11.010870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:11.010870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:11.010870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:11.010870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:11.012474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:11.012474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:11.012474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:11.012474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:11.012474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:11.012474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:11.012474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:11.012474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:11.012474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:11.012474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:11.012474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:11.012474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:11.012474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 332/662] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:00:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:11.012474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7e69c-e619-4536-bd21-d3911c78524a None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:11.081845 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 331/663] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:00:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:00:11.082754 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067348cc-9116-4dca-99c0-a70306e9b6f2 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:11.103147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:11.131881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7e69c-e619-4536-bd21-d3911c78524a None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:11.153562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:11.168291 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b82122-1f54-4412-b7ab-1e36e09164f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:11.169095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b82122-1f54-4412-b7ab-1e36e09164f3 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:11.169372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b82122-1f54-4412-b7ab-1e36e09164f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:11.170285 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b82122-1f54-4412-b7ab-1e36e09164f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:11.179131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b82122-1f54-4412-b7ab-1e36e09164f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:11.180587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b82122-1f54-4412-b7ab-1e36e09164f3 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:11.195475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-c6b82122-1f54-4412-b7ab-1e36e09164f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:11.195475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:11.195475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:11.195475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:11.195475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:11.195475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:11.195475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:11.195475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:11.195475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:11.195475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:11.195475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:11.195475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:11.195475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:11.195475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:11.195475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:11.196409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:11.197299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:11.197299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:11.197299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:11.197299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:11.197299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:11.197299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:11.197299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:11.197299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:11.197299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 333/664] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:11.197796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b82122-1f54-4412-b7ab-1e36e09164f3 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:11.208029 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067348cc-9116-4dca-99c0-a70306e9b6f2 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:11.245530 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46808aa2-d142-48ba-a7fc-981d6f3cdc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:11.246251 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46808aa2-d142-48ba-a7fc-981d6f3cdc57 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:11.246534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46808aa2-d142-48ba-a7fc-981d6f3cdc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:11.246841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46808aa2-d142-48ba-a7fc-981d6f3cdc57 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:11.254617 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46808aa2-d142-48ba-a7fc-981d6f3cdc57 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:11.256116 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46808aa2-d142-48ba-a7fc-981d6f3cdc57 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:11.272089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-46808aa2-d142-48ba-a7fc-981d6f3cdc57 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 09:00:11.272089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:11.272089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:11.272089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:11.272089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:11.272089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:11.272089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:11.272089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:11.272089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:11.272089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:11.272089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:11.272089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:11.272089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:11.272089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:11.272089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:11.272089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:11.273068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:11.274036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:11.274036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:11.274036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:11.274036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:11.274036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:11.274036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:11.274036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:11.274036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:11.274036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 09:00:11.274036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:11.274036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 332/665] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:00:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:11.278564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46808aa2-d142-48ba-a7fc-981d6f3cdc57 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:11.314325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b82122-1f54-4412-b7ab-1e36e09164f3 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:11.350882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8874051-ac10-4d07-bcc5-9dfe618c6f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:11.351530 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8874051-ac10-4d07-bcc5-9dfe618c6f99 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:11.352000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8874051-ac10-4d07-bcc5-9dfe618c6f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:11.352279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8874051-ac10-4d07-bcc5-9dfe618c6f99 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:11.353022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8874051-ac10-4d07-bcc5-9dfe618c6f99 None admin] RBAC: Adding query filter params (name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:11.353327 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8874051-ac10-4d07-bcc5-9dfe618c6f99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:11.354772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8874051-ac10-4d07-bcc5-9dfe618c6f99 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:11.362521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:11.370841 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 334/666] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 09:00:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:11.371856 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8874051-ac10-4d07-bcc5-9dfe618c6f99 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:11.394572 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46808aa2-d142-48ba-a7fc-981d6f3cdc57 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:11.414372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:11.434368 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa685032-8eeb-4ae6-953c-97541ca097db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:11.435537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa685032-8eeb-4ae6-953c-97541ca097db None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:11.435821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa685032-8eeb-4ae6-953c-97541ca097db None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:11.436092 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa685032-8eeb-4ae6-953c-97541ca097db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:11.447590 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa685032-8eeb-4ae6-953c-97541ca097db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:11.449085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa685032-8eeb-4ae6-953c-97541ca097db None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:11.471131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-aa685032-8eeb-4ae6-953c-97541ca097db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:11.471131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:11.471131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:11.471131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:11.471131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:11.471131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:11.471131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:11.471131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:11.471131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:11.471131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:11.471131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:11.471131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:11.471131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:11.471131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:11.471131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:11.472110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:11.472986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:11.472986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:11.472986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:11.472986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:11.472986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 333/667] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:11.473705 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa685032-8eeb-4ae6-953c-97541ca097db None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:11.492375 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8874051-ac10-4d07-bcc5-9dfe618c6f99 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:11.529620 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f8c9b-f96d-471b-a6c7-bc3135ae1c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:11.530526 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f8c9b-f96d-471b-a6c7-bc3135ae1c68 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:11.530983 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f8c9b-f96d-471b-a6c7-bc3135ae1c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:11.531421 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f8c9b-f96d-471b-a6c7-bc3135ae1c68 None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:11.532400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f8c9b-f96d-471b-a6c7-bc3135ae1c68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:11.532880 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f8c9b-f96d-471b-a6c7-bc3135ae1c68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:11.534354 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f8c9b-f96d-471b-a6c7-bc3135ae1c68 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:11.553997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 335/668] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:11.554870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f8c9b-f96d-471b-a6c7-bc3135ae1c68 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:11.592671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa685032-8eeb-4ae6-953c-97541ca097db None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:11.622475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:11.628789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d25c97-e939-4ad3-87ee-db90061df427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:11.629462 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25c97-e939-4ad3-87ee-db90061df427 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:11.629750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25c97-e939-4ad3-87ee-db90061df427 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:11.630032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25c97-e939-4ad3-87ee-db90061df427 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:11.630901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d25c97-e939-4ad3-87ee-db90061df427 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:11.631265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d25c97-e939-4ad3-87ee-db90061df427 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:11.632741 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d25c97-e939-4ad3-87ee-db90061df427 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:11.655311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 334/669] 10.222.0.44 () {56 vars in 1139 bytes} [Wed Jul 20 09:00:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:11.655311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d25c97-e939-4ad3-87ee-db90061df427 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:11.672748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f8c9b-f96d-471b-a6c7-bc3135ae1c68 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:11.684652 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:11.707830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1daf0b-c562-48b3-88d3-c804a061e36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:11.708621 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1daf0b-c562-48b3-88d3-c804a061e36b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:11.708964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1daf0b-c562-48b3-88d3-c804a061e36b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:11.709355 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1daf0b-c562-48b3-88d3-c804a061e36b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:11.719663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1daf0b-c562-48b3-88d3-c804a061e36b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:11.721110 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1daf0b-c562-48b3-88d3-c804a061e36b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:11.741351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-ff1daf0b-c562-48b3-88d3-c804a061e36b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:11.741351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:11.741351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:11.741351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:11.741351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:11.741351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:11.741351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:11.741351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:11.741351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:11.741351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:11.741351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:11.741351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:11.741351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:11.741351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:11.741351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:11.742465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:11.743570 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:11.743570 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:11.743570 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:11.743570 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:11.747125 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 336/670] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:11.748317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1daf0b-c562-48b3-88d3-c804a061e36b None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:11.766443 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d25c97-e939-4ad3-87ee-db90061df427 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:11.803264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d2a76-4135-42a4-9118-a12e4f0f5fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:11.803841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d2a76-4135-42a4-9118-a12e4f0f5fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:11.804121 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d2a76-4135-42a4-9118-a12e4f0f5fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:11.804394 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d2a76-4135-42a4-9118-a12e4f0f5fa3 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:11.805109 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d2a76-4135-42a4-9118-a12e4f0f5fa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:11.805411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d2a76-4135-42a4-9118-a12e4f0f5fa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:11.806783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d2a76-4135-42a4-9118-a12e4f0f5fa3 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:11.819842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 335/671] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:11.820477 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d2a76-4135-42a4-9118-a12e4f0f5fa3 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:11.856576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1daf0b-c562-48b3-88d3-c804a061e36b None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:11.867622 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:11.892741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cdc2e-0d42-40c7-a128-aa302e754e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:11.893404 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cdc2e-0d42-40c7-a128-aa302e754e1b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:11.893692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cdc2e-0d42-40c7-a128-aa302e754e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:11.893974 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cdc2e-0d42-40c7-a128-aa302e754e1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:11.894847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cdc2e-0d42-40c7-a128-aa302e754e1b None admin] RBAC: Adding query filter params (role.id=ae949d8bea4b40d9b1a9fa207d539663, scope.project.id=a498284274454d25b1b118a88996e522, user.id=bb93eb1d25c4410591125b0ab7a73149) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:11.895163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cdc2e-0d42-40c7-a128-aa302e754e1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:11.896603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cdc2e-0d42-40c7-a128-aa302e754e1b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:11.902644 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 337/672] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:00:11 2022] GET /identity/v3/role_assignments?user.id=bb93eb1d25c4410591125b0ab7a73149&scope.project.id=a498284274454d25b1b118a88996e522&role.id=ae949d8bea4b40d9b1a9fa207d539663 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:11.903451 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cdc2e-0d42-40c7-a128-aa302e754e1b None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:11.925416 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:11.941359 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d2a76-4135-42a4-9118-a12e4f0f5fa3 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:11.989486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043328b8-f185-4315-9fea-9e7f7cf8b364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:11.990321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043328b8-f185-4315-9fea-9e7f7cf8b364 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:11.990609 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043328b8-f185-4315-9fea-9e7f7cf8b364 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:11.990925 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043328b8-f185-4315-9fea-9e7f7cf8b364 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:12.031574 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cdc2e-0d42-40c7-a128-aa302e754e1b None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:12.072874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288e5d8-4a66-4fd7-8c00-1a0ca9b70bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:12.073910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288e5d8-4a66-4fd7-8c00-1a0ca9b70bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:12.074403 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288e5d8-4a66-4fd7-8c00-1a0ca9b70bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:12.074923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288e5d8-4a66-4fd7-8c00-1a0ca9b70bc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:12.075960 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288e5d8-4a66-4fd7-8c00-1a0ca9b70bc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:12.076492 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288e5d8-4a66-4fd7-8c00-1a0ca9b70bc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:12.078482 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288e5d8-4a66-4fd7-8c00-1a0ca9b70bc5 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:12.086028 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043328b8-f185-4315-9fea-9e7f7cf8b364 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:12.087531 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043328b8-f185-4315-9fea-9e7f7cf8b364 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:12.095953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 338/673] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:12.097952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288e5d8-4a66-4fd7-8c00-1a0ca9b70bc5 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:12.143092 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:12.192324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:12.214712 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 336/674] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:00:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:00:12.214712 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043328b8-f185-4315-9fea-9e7f7cf8b364 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:12.233747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288e5d8-4a66-4fd7-8c00-1a0ca9b70bc5 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:12.282191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9154349-1be2-4180-8f64-c595769f61d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:12.283269 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9154349-1be2-4180-8f64-c595769f61d1 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:12.283556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9154349-1be2-4180-8f64-c595769f61d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:12.283835 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9154349-1be2-4180-8f64-c595769f61d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:12.293603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9154349-1be2-4180-8f64-c595769f61d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:12.296501 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9154349-1be2-4180-8f64-c595769f61d1 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:12.316636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-d9154349-1be2-4180-8f64-c595769f61d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:12.316636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:12.316636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:12.316636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:12.316636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:12.316636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:12.316636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:12.316636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:12.316636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:12.316636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:12.316636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:12.316636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:12.316636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:12.316636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:12.316636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:12.317578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:12.318480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:12.318480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:12.318480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:12.318480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:12.318480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:12.318480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:12.318480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:12.318480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:12.318480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 339/675] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:12.319014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9154349-1be2-4180-8f64-c595769f61d1 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:12.338542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043328b8-f185-4315-9fea-9e7f7cf8b364 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:12.387777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709829d3-c019-4389-8497-f339e468bc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:12.390088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709829d3-c019-4389-8497-f339e468bc93 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:12.390591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709829d3-c019-4389-8497-f339e468bc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:12.391242 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709829d3-c019-4389-8497-f339e468bc93 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:12.399935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709829d3-c019-4389-8497-f339e468bc93 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:12.401594 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709829d3-c019-4389-8497-f339e468bc93 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:12.417088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-709829d3-c019-4389-8497-f339e468bc93 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:00:12.417088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:12.417088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:12.417088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:12.417088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:12.417088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:12.417088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:12.417088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:12.417088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:12.417088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:12.417088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:12.417088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:12.417088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:12.417088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:12.417088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:12.417088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:12.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:12.419496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:12.419496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:12.419496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:12.419496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:12.419496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:12.419496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:12.419496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:12.419496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:12.419496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:00:12.419496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:12.419496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 337/676] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:00:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:12.420599 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709829d3-c019-4389-8497-f339e468bc93 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:12.447155 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:12.457193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9154349-1be2-4180-8f64-c595769f61d1 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:12.471607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:12.492904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d457755-aaf5-4dbb-a521-5d1ad4e8914e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:12.493687 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d457755-aaf5-4dbb-a521-5d1ad4e8914e None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:12.493971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d457755-aaf5-4dbb-a521-5d1ad4e8914e None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:12.494253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d457755-aaf5-4dbb-a521-5d1ad4e8914e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:12.505522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d457755-aaf5-4dbb-a521-5d1ad4e8914e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:12.507006 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d457755-aaf5-4dbb-a521-5d1ad4e8914e None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:12.524810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-3d457755-aaf5-4dbb-a521-5d1ad4e8914e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:12.524810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:12.524810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:12.524810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:12.524810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:12.524810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:12.524810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:12.524810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:12.524810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:12.524810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:12.524810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:12.524810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:12.524810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:12.524810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:12.524810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:12.525929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:12.526874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:12.526874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:12.526874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:12.526874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:12.526874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 340/677] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:12.527573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6348cff-9028-4eac-86df-efea78934d04 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:12.527840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6348cff-9028-4eac-86df-efea78934d04 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:12.528188 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6348cff-9028-4eac-86df-efea78934d04 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:12.533911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 341/678] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:12.533911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6348cff-9028-4eac-86df-efea78934d04 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:12.542157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709829d3-c019-4389-8497-f339e468bc93 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:12.586048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd1505-f3c2-4559-b10f-de0691a3d30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:12.587030 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd1505-f3c2-4559-b10f-de0691a3d30f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:12.587320 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd1505-f3c2-4559-b10f-de0691a3d30f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:12.587600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd1505-f3c2-4559-b10f-de0691a3d30f None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:12.588390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd1505-f3c2-4559-b10f-de0691a3d30f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:12.588753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd1505-f3c2-4559-b10f-de0691a3d30f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:12.590582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd1505-f3c2-4559-b10f-de0691a3d30f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:12.603126 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 338/679] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:12.604006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd1505-f3c2-4559-b10f-de0691a3d30f None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:12.644112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6348cff-9028-4eac-86df-efea78934d04 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:12.679382 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9fd3ed-6cf4-421c-bb08-5167ebe5f932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:12.680043 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fd3ed-6cf4-421c-bb08-5167ebe5f932 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:12.680326 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fd3ed-6cf4-421c-bb08-5167ebe5f932 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:12.680599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fd3ed-6cf4-421c-bb08-5167ebe5f932 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:12.681419 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9fd3ed-6cf4-421c-bb08-5167ebe5f932 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:12.681723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9fd3ed-6cf4-421c-bb08-5167ebe5f932 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:12.683203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9fd3ed-6cf4-421c-bb08-5167ebe5f932 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:12.702560 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 342/680] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Jul 20 09:00:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:12.703732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99eca69-56c1-4c40-902b-06b7beca8f69 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:12.704020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99eca69-56c1-4c40-902b-06b7beca8f69 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:12.704296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99eca69-56c1-4c40-902b-06b7beca8f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:12.710024 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:12.718428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd1505-f3c2-4559-b10f-de0691a3d30f None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:12.723396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-f99eca69-56c1-4c40-902b-06b7beca8f69 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:12.724585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f99eca69-56c1-4c40-902b-06b7beca8f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:12.727525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:12.756045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4099f4d-a3ac-4edf-8919-1e6999003d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:12.756710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4099f4d-a3ac-4edf-8919-1e6999003d9a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:12.756992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4099f4d-a3ac-4edf-8919-1e6999003d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:12.757269 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4099f4d-a3ac-4edf-8919-1e6999003d9a None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:12.757990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4099f4d-a3ac-4edf-8919-1e6999003d9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:12.758290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4099f4d-a3ac-4edf-8919-1e6999003d9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:12.759784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4099f4d-a3ac-4edf-8919-1e6999003d9a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:12.772905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 339/681] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:12.773568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4099f4d-a3ac-4edf-8919-1e6999003d9a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:12.807010 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 343/682] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:12.807010 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99eca69-56c1-4c40-902b-06b7beca8f69 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:12.881043 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4099f4d-a3ac-4edf-8919-1e6999003d9a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:12.917018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b23f5e-45b1-4e89-bbfe-09f3b9f338cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:12.917669 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b23f5e-45b1-4e89-bbfe-09f3b9f338cb None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:12.917950 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b23f5e-45b1-4e89-bbfe-09f3b9f338cb None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:12.918224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b23f5e-45b1-4e89-bbfe-09f3b9f338cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:12.919080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b23f5e-45b1-4e89-bbfe-09f3b9f338cb None admin] RBAC: Adding query filter params (role.id=ae949d8bea4b40d9b1a9fa207d539663, scope.project.id=a498284274454d25b1b118a88996e522, user.id=99dc3db93e9045b8b3ef59a030822972) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:12.919436 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b23f5e-45b1-4e89-bbfe-09f3b9f338cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:12.920020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99eca69-56c1-4c40-902b-06b7beca8f69 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:12.920846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b23f5e-45b1-4e89-bbfe-09f3b9f338cb None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:12.934713 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 340/683] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:00:12 2022] GET /identity/v3/role_assignments?user.id=99dc3db93e9045b8b3ef59a030822972&scope.project.id=a498284274454d25b1b118a88996e522&role.id=ae949d8bea4b40d9b1a9fa207d539663 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:12.934713 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b23f5e-45b1-4e89-bbfe-09f3b9f338cb None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:12.962605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c282f-d9d3-4848-9112-2b8aad1019e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:12.963510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c282f-d9d3-4848-9112-2b8aad1019e9 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:12.964111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c282f-d9d3-4848-9112-2b8aad1019e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:12.964398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c282f-d9d3-4848-9112-2b8aad1019e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:12.968203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:12.983495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:13.063832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c282f-d9d3-4848-9112-2b8aad1019e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:13.066028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c282f-d9d3-4848-9112-2b8aad1019e9 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:13.076420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b23f5e-45b1-4e89-bbfe-09f3b9f338cb None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:13.129532 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803349eb-7398-46d2-9cca-edef750c6a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:13.130628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803349eb-7398-46d2-9cca-edef750c6a49 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:13.131101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803349eb-7398-46d2-9cca-edef750c6a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:13.131328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803349eb-7398-46d2-9cca-edef750c6a49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:13.141399 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803349eb-7398-46d2-9cca-edef750c6a49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:13.142117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803349eb-7398-46d2-9cca-edef750c6a49 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:13.157761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-803349eb-7398-46d2-9cca-edef750c6a49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:13.157761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:13.157761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:13.157761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:13.157761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:13.157761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:13.157761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:13.157761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:13.157761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:13.157761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:13.157761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:13.157761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:13.157761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:13.157761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:13.157761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:13.158936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:13.159907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:13.159907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:13.159907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:13.159907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:13.159907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:13.159907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:13.159907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:13.159907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:13.159907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 341/684] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:13.160413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803349eb-7398-46d2-9cca-edef750c6a49 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:13.211477 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 344/685] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:00:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:00:13.212553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c282f-d9d3-4848-9112-2b8aad1019e9 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:13.249541 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:13.288720 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:13.299295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803349eb-7398-46d2-9cca-edef750c6a49 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:13.333405 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c282f-d9d3-4848-9112-2b8aad1019e9 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:13.335562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8526d-a004-444d-9c60-6a316f3a32a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:13.336469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8526d-a004-444d-9c60-6a316f3a32a3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:13.336889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8526d-a004-444d-9c60-6a316f3a32a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:13.337271 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8526d-a004-444d-9c60-6a316f3a32a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:13.351320 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8526d-a004-444d-9c60-6a316f3a32a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:13.362289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-6ba8526d-a004-444d-9c60-6a316f3a32a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:00:13.362289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:13.362289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:13.362289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:13.362289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:13.362289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:13.362289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:13.362289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:13.362289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:13.362289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:13.362289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:13.362289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:13.362289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:13.362289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:13.362289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:13.362289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:13.363647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:13.363647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:13.363647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:13.363647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:13.363647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:13.363647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:13.363647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:13.363647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:13.363647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:13.363647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:13.363647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:13.363647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:13.363647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:13.363647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:13.363647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:13.363647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:13.365498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:13.365498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:13.365498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:13.365498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:13.365498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:13.365498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:13.365498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:13.365498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:13.365498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:13.365498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:13.365498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:00:13.365498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:13.365498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 342/686] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:00:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:13.365498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8526d-a004-444d-9c60-6a316f3a32a3 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:13.373309 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecde8a-dcc5-4c59-abf5-99fa4e5c1592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:13.374083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecde8a-dcc5-4c59-abf5-99fa4e5c1592 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:13.374472 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecde8a-dcc5-4c59-abf5-99fa4e5c1592 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:13.374882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecde8a-dcc5-4c59-abf5-99fa4e5c1592 None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:13.375889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecde8a-dcc5-4c59-abf5-99fa4e5c1592 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:13.376303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecde8a-dcc5-4c59-abf5-99fa4e5c1592 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:13.377950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecde8a-dcc5-4c59-abf5-99fa4e5c1592 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:13.391918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 345/687] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:13.392928 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecde8a-dcc5-4c59-abf5-99fa4e5c1592 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:13.492544 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8526d-a004-444d-9c60-6a316f3a32a3 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:13.496338 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:13.508943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecde8a-dcc5-4c59-abf5-99fa4e5c1592 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:13.538645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12396a8-4c57-4c1c-8162-69eba0b867d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:13.539512 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12396a8-4c57-4c1c-8162-69eba0b867d7 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:13.539801 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12396a8-4c57-4c1c-8162-69eba0b867d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:13.540337 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12396a8-4c57-4c1c-8162-69eba0b867d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:13.554715 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12396a8-4c57-4c1c-8162-69eba0b867d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:13.556166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12396a8-4c57-4c1c-8162-69eba0b867d7 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:13.563952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b581b-3ce5-4548-9cc3-6f8732b6bb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:13.564640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b581b-3ce5-4548-9cc3-6f8732b6bb71 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:13.564923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b581b-3ce5-4548-9cc3-6f8732b6bb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:13.565206 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b581b-3ce5-4548-9cc3-6f8732b6bb71 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:13.566015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b581b-3ce5-4548-9cc3-6f8732b6bb71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:13.566423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b581b-3ce5-4548-9cc3-6f8732b6bb71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:13.567761 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b581b-3ce5-4548-9cc3-6f8732b6bb71 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:13.576556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-e12396a8-4c57-4c1c-8162-69eba0b867d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:13.576556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:13.576556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:13.576556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:13.576556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:13.576556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:13.576556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:13.576556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:13.576556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:13.576556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:13.576556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:13.576556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:13.576556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:13.576556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:13.576556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:13.577490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:13.578430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:13.578430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:13.578430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:13.578430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:13.578430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 343/688] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:13.579171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12396a8-4c57-4c1c-8162-69eba0b867d7 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:13.581537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 346/689] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:00:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:13.586522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b581b-3ce5-4548-9cc3-6f8732b6bb71 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:13.593249 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:13.702942 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b581b-3ce5-4548-9cc3-6f8732b6bb71 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:13.731253 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12396a8-4c57-4c1c-8162-69eba0b867d7 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:13.748972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c74f38-24ca-4bff-9924-19c7824d0c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:13.750341 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c74f38-24ca-4bff-9924-19c7824d0c71 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:13.750884 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c74f38-24ca-4bff-9924-19c7824d0c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:13.751627 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c74f38-24ca-4bff-9924-19c7824d0c71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:13.766496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c74f38-24ca-4bff-9924-19c7824d0c71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:13.768471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c74f38-24ca-4bff-9924-19c7824d0c71 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:13.769668 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03abeb-e466-491a-8352-0b3a762f69e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:13.770401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03abeb-e466-491a-8352-0b3a762f69e3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:13.770782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03abeb-e466-491a-8352-0b3a762f69e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:13.771177 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03abeb-e466-491a-8352-0b3a762f69e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:13.772100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03abeb-e466-491a-8352-0b3a762f69e3 None admin] RBAC: Adding query filter params (role.id=ae949d8bea4b40d9b1a9fa207d539663, scope.project.id=a498284274454d25b1b118a88996e522, user.id=5e5ad4e122ae471cad4cf1b0e3111dd9) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:13.772486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03abeb-e466-491a-8352-0b3a762f69e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:13.773971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03abeb-e466-491a-8352-0b3a762f69e3 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:13.781359 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 344/690] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:00:13 2022] GET /identity/v3/role_assignments?user.id=5e5ad4e122ae471cad4cf1b0e3111dd9&scope.project.id=a498284274454d25b1b118a88996e522&role.id=ae949d8bea4b40d9b1a9fa207d539663 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:13.782317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03abeb-e466-491a-8352-0b3a762f69e3 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:13.790990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-c1c74f38-24ca-4bff-9924-19c7824d0c71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:13.790990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:13.790990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:13.790990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:13.790990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:13.790990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:13.790990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:13.790990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:13.790990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:13.790990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:13.790990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:13.790990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:13.790990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:13.790990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:13.790990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:13.791991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:13.793339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:13.793339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:13.793339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:13.793339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:13.793339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 347/691] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:13.799918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c74f38-24ca-4bff-9924-19c7824d0c71 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:13.813370 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:13.900514 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:13.922948 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c74f38-24ca-4bff-9924-19c7824d0c71 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:13.945500 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03abeb-e466-491a-8352-0b3a762f69e3 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:13.963558 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981c5ab-5237-4242-b878-af57e27f4304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:13.964263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981c5ab-5237-4242-b878-af57e27f4304 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:13.965045 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981c5ab-5237-4242-b878-af57e27f4304 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:13.965401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981c5ab-5237-4242-b878-af57e27f4304 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:13.966522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981c5ab-5237-4242-b878-af57e27f4304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:13.967416 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981c5ab-5237-4242-b878-af57e27f4304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:13.968894 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981c5ab-5237-4242-b878-af57e27f4304 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:13.984071 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 348/692] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:13.991324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981c5ab-5237-4242-b878-af57e27f4304 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:14.005662 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347fd51-a684-462b-8cb1-978823b26d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:14.006352 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347fd51-a684-462b-8cb1-978823b26d64 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:14.006636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347fd51-a684-462b-8cb1-978823b26d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:14.007046 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347fd51-a684-462b-8cb1-978823b26d64 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:14.007789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347fd51-a684-462b-8cb1-978823b26d64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:14.008098 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347fd51-a684-462b-8cb1-978823b26d64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:14.009479 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347fd51-a684-462b-8cb1-978823b26d64 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:14.040133 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 345/693] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:14.046917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347fd51-a684-462b-8cb1-978823b26d64 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:14.129840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:14.177263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981c5ab-5237-4242-b878-af57e27f4304 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:14.195278 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347fd51-a684-462b-8cb1-978823b26d64 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:14.230778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fbd81-a96f-4962-9ff2-228d4ee343cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:14.230778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fbd81-a96f-4962-9ff2-228d4ee343cf None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:14.230778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fbd81-a96f-4962-9ff2-228d4ee343cf None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:14.230778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fbd81-a96f-4962-9ff2-228d4ee343cf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:14.231655 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:14.237993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae560603-b927-462f-a17a-1555ebbb3fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:14.238841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae560603-b927-462f-a17a-1555ebbb3fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:14.239127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae560603-b927-462f-a17a-1555ebbb3fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:14.239405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae560603-b927-462f-a17a-1555ebbb3fc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:14.239609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140fbd81-a96f-4962-9ff2-228d4ee343cf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:14.241231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140fbd81-a96f-4962-9ff2-228d4ee343cf None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:14.246455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae560603-b927-462f-a17a-1555ebbb3fc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:14.248635 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae560603-b927-462f-a17a-1555ebbb3fc6 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:14.257179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-140fbd81-a96f-4962-9ff2-228d4ee343cf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:00:14.257179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:14.257179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:14.257179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:14.257179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:14.257179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:14.257179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:14.257179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:14.257179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:14.257179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:14.257179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:14.257179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:14.257179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:14.257179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:14.257179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:14.257179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:14.258303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:14.260335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:14.260335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:14.260335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:14.260335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:14.260335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:14.260335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:14.260335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:14.260335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:14.260335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:00:14.260335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:14.260335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 349/694] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jul 20 09:00:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:14.261662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575cbbc-379f-4944-b489-9cfac7c0c136 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:14.261662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575cbbc-379f-4944-b489-9cfac7c0c136 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:14.261662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575cbbc-379f-4944-b489-9cfac7c0c136 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:14.266117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-ae560603-b927-462f-a17a-1555ebbb3fc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:14.266117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:14.266117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:14.266117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:14.266117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:14.266117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:14.266117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:14.266117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:14.266117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:14.266117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:14.266117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:14.266117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:14.266117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:14.266117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:14.266117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:14.267582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:14.268681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:14.268681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:14.268681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:14.268681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:14.268681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:14.268681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:14.268681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:14.268681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:14.269190 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 350/695] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:14.274744 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 346/696] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:14.274744 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae560603-b927-462f-a17a-1555ebbb3fc6 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:14.275003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb99104-9cb3-4be4-a069-3430b9962f6c None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:14.275003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb99104-9cb3-4be4-a069-3430b9962f6c None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:14.275003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb99104-9cb3-4be4-a069-3430b9962f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:14.296019 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-bdb99104-9cb3-4be4-a069-3430b9962f6c None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:14.298425 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-bdb99104-9cb3-4be4-a069-3430b9962f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:14.376550 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 351/697] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:14.377799 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb99104-9cb3-4be4-a069-3430b9962f6c None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:14.397874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae560603-b927-462f-a17a-1555ebbb3fc6 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:14.449526 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:14.453129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d2e17-543c-4dca-86a0-ea42cd868234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:14.454019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d2e17-543c-4dca-86a0-ea42cd868234 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:14.454643 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d2e17-543c-4dca-86a0-ea42cd868234 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:14.455048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d2e17-543c-4dca-86a0-ea42cd868234 None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:14.456387 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d2e17-543c-4dca-86a0-ea42cd868234 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:14.457221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d2e17-543c-4dca-86a0-ea42cd868234 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:14.458742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d2e17-543c-4dca-86a0-ea42cd868234 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:14.480436 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 347/698] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:14.480436 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d2e17-543c-4dca-86a0-ea42cd868234 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:14.503160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb99104-9cb3-4be4-a069-3430b9962f6c None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:14.541074 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:14.542029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db423d24-4545-47b2-92dc-4c50fc0a3076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:14.543150 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db423d24-4545-47b2-92dc-4c50fc0a3076 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:14.545119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db423d24-4545-47b2-92dc-4c50fc0a3076 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:14.545119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db423d24-4545-47b2-92dc-4c50fc0a3076 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:14.545803 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db423d24-4545-47b2-92dc-4c50fc0a3076 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:14.546380 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db423d24-4545-47b2-92dc-4c50fc0a3076 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:14.548190 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db423d24-4545-47b2-92dc-4c50fc0a3076 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:14.574841 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 352/699] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jul 20 09:00:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:14.576426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db423d24-4545-47b2-92dc-4c50fc0a3076 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:14.610341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d2e17-543c-4dca-86a0-ea42cd868234 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:14.647416 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e33e3-40c9-403c-a1df-452268eb072f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:14.648372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e33e3-40c9-403c-a1df-452268eb072f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:14.648759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e33e3-40c9-403c-a1df-452268eb072f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:14.649141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e33e3-40c9-403c-a1df-452268eb072f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:14.660870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e33e3-40c9-403c-a1df-452268eb072f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:14.662455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e33e3-40c9-403c-a1df-452268eb072f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:14.681428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-316e33e3-40c9-403c-a1df-452268eb072f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:14.681428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:14.681428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:14.681428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:14.681428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:14.681428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:14.681428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:14.681428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:14.681428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:14.681428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:14.681428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:14.681428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:14.681428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:14.681428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:14.681428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:14.682555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:14.683696 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:14.683696 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:14.683696 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:14.683696 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:14.689997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 348/700] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:14.691099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e33e3-40c9-403c-a1df-452268eb072f None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:14.709778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db423d24-4545-47b2-92dc-4c50fc0a3076 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:14.728313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:14.749862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6c30d-3a24-4c42-8d13-bc86fd098331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:14.750650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6c30d-3a24-4c42-8d13-bc86fd098331 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:14.751039 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6c30d-3a24-4c42-8d13-bc86fd098331 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:14.751392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6c30d-3a24-4c42-8d13-bc86fd098331 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:14.752302 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6c30d-3a24-4c42-8d13-bc86fd098331 None admin] RBAC: Adding query filter params (role.id=ae949d8bea4b40d9b1a9fa207d539663, scope.project.id=a498284274454d25b1b118a88996e522, user.id=03b4375e6ead42fb8d77a7b30617cb95) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:14.752687 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6c30d-3a24-4c42-8d13-bc86fd098331 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:14.754276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab6c30d-3a24-4c42-8d13-bc86fd098331 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:14.766842 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 353/701] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:00:14 2022] GET /identity/v3/role_assignments?user.id=03b4375e6ead42fb8d77a7b30617cb95&scope.project.id=a498284274454d25b1b118a88996e522&role.id=ae949d8bea4b40d9b1a9fa207d539663 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:14.767923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6c30d-3a24-4c42-8d13-bc86fd098331 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:14.822012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:14.827231 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e33e3-40c9-403c-a1df-452268eb072f None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:14.878129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcf659-0add-4144-87d9-e1fa7183a14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:14.878841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcf659-0add-4144-87d9-e1fa7183a14f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:14.879128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcf659-0add-4144-87d9-e1fa7183a14f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:14.879408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcf659-0add-4144-87d9-e1fa7183a14f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:14.971608 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6c30d-3a24-4c42-8d13-bc86fd098331 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:15.017825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcf659-0add-4144-87d9-e1fa7183a14f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:15.019360 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcf659-0add-4144-87d9-e1fa7183a14f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:15.022235 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227e2a0-2957-4114-9d36-cf89bccbe94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:15.024452 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227e2a0-2957-4114-9d36-cf89bccbe94b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:15.024749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227e2a0-2957-4114-9d36-cf89bccbe94b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:15.025029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227e2a0-2957-4114-9d36-cf89bccbe94b None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:15.025767 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227e2a0-2957-4114-9d36-cf89bccbe94b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:15.026075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227e2a0-2957-4114-9d36-cf89bccbe94b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:15.027492 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b227e2a0-2957-4114-9d36-cf89bccbe94b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:15.045547 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 354/702] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:15.051374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227e2a0-2957-4114-9d36-cf89bccbe94b None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:15.109378 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:15.148486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:15.152644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 349/703] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:00:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:00:15.165266 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcf659-0add-4144-87d9-e1fa7183a14f None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:15.195737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227e2a0-2957-4114-9d36-cf89bccbe94b None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:15.240995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35709a-6ac1-4925-8a53-39b56f848a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:15.241766 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35709a-6ac1-4925-8a53-39b56f848a70 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:15.241964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35709a-6ac1-4925-8a53-39b56f848a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:15.242220 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35709a-6ac1-4925-8a53-39b56f848a70 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:15.242996 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35709a-6ac1-4925-8a53-39b56f848a70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:15.244508 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35709a-6ac1-4925-8a53-39b56f848a70 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:15.271521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 355/704] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 09:00:15 2022] POST /identity/v3/users => generated 254 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:00:15.273376 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3ba76-d7fb-450a-b231-efb441a1988d None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:15.273908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3ba76-d7fb-450a-b231-efb441a1988d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:15.274395 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3ba76-d7fb-450a-b231-efb441a1988d None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:15.276334 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 356/705] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:15.288291 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e165f76-7dbe-4599-84f5-9e77ed76f7ba None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:15.288929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e165f76-7dbe-4599-84f5-9e77ed76f7ba None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:15.289444 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e165f76-7dbe-4599-84f5-9e77ed76f7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:15.318120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcf659-0add-4144-87d9-e1fa7183a14f None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:15.321291 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-5e165f76-7dbe-4599-84f5-9e77ed76f7ba None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:15.322526 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5e165f76-7dbe-4599-84f5-9e77ed76f7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:15.393277 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9ef71-349f-4ba6-8383-417e04dcb1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:15.394097 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9ef71-349f-4ba6-8383-417e04dcb1db None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:15.395044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9ef71-349f-4ba6-8383-417e04dcb1db None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:15.395335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9ef71-349f-4ba6-8383-417e04dcb1db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:15.431979 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9ef71-349f-4ba6-8383-417e04dcb1db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:15.433865 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9ef71-349f-4ba6-8383-417e04dcb1db None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:15.436191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 357/706] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:15.460074 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e165f76-7dbe-4599-84f5-9e77ed76f7ba None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:15.469526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-05c9ef71-349f-4ba6-8383-417e04dcb1db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:15.469526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:15.469526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:15.469526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:15.469526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:15.469526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:15.469526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:15.469526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:15.469526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:15.469526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:15.469526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:15.469526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:15.469526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:15.469526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:15.469526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:15.470420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:15.471338 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:15.471338 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:15.471338 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:15.471338 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:15.474894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 350/707] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:15.476885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9ef71-349f-4ba6-8383-417e04dcb1db None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:15.494555 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:15.546521 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:15.602304 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e165f76-7dbe-4599-84f5-9e77ed76f7ba None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:15.605854 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9ef71-349f-4ba6-8383-417e04dcb1db None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:15.645432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89ffc1-1336-4375-995d-fa26f260c480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:15.646129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89ffc1-1336-4375-995d-fa26f260c480 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:15.646415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89ffc1-1336-4375-995d-fa26f260c480 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:15.647383 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401780d-341a-478d-9933-3669ff93d5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:15.648091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac89ffc1-1336-4375-995d-fa26f260c480 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:15.648271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401780d-341a-478d-9933-3669ff93d5ce None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:15.648520 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401780d-341a-478d-9933-3669ff93d5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:15.648817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401780d-341a-478d-9933-3669ff93d5ce None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:15.648935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac89ffc1-1336-4375-995d-fa26f260c480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:15.649141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac89ffc1-1336-4375-995d-fa26f260c480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:15.650512 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac89ffc1-1336-4375-995d-fa26f260c480 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:15.651426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401780d-341a-478d-9933-3669ff93d5ce None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:15.652896 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401780d-341a-478d-9933-3669ff93d5ce None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:15.671560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 351/708] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:15.672770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-e401780d-341a-478d-9933-3669ff93d5ce None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:00:15.672770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:15.672770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:15.672770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:15.672770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:15.672770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:15.672770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:15.672770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:15.672770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:00:15.672770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:15.672770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:15.672770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:15.672770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:15.672770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:15.672770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:15.672770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:15.673847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:15.673847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:15.673847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:15.673847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:15.673847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:15.673847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:15.673847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:15.673847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:15.673847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:15.673847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:15.673847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:15.673847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:15.673847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:15.673847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:15.673847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:15.673847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:15.675037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:15.675037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:15.675037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:15.675037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:15.675037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:15.675037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:15.675037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:15.675037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:15.675037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:15.675037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:15.675037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:15.675037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:15.675037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:15.675037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:00:15.675037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:15.675037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 358/709] 10.222.0.44 () {56 vars in 1129 bytes} [Wed Jul 20 09:00:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:15.681146 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89ffc1-1336-4375-995d-fa26f260c480 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:15.683624 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401780d-341a-478d-9933-3669ff93d5ce None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:15.815328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:15.833516 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401780d-341a-478d-9933-3669ff93d5ce None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:15.888713 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac89ffc1-1336-4375-995d-fa26f260c480 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:15.903267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:15.930744 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6c47f-f73c-4bad-bf45-ee8345cd3b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:15.930744 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6c47f-f73c-4bad-bf45-ee8345cd3b2a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:15.930744 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6c47f-f73c-4bad-bf45-ee8345cd3b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:15.931111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6c47f-f73c-4bad-bf45-ee8345cd3b2a None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:15.933085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6c47f-f73c-4bad-bf45-ee8345cd3b2a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:15.933085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6c47f-f73c-4bad-bf45-ee8345cd3b2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:15.934786 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6c47f-f73c-4bad-bf45-ee8345cd3b2a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:15.949625 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 352/710] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Jul 20 09:00:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:15.955335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6c47f-f73c-4bad-bf45-ee8345cd3b2a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:15.994736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb3fc7-93c6-46e2-822d-13af89260e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:15.994736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3fc7-93c6-46e2-822d-13af89260e41 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:15.994736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3fc7-93c6-46e2-822d-13af89260e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:15.994736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb3fc7-93c6-46e2-822d-13af89260e41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:16.022726 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb3fc7-93c6-46e2-822d-13af89260e41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:16.022726 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb3fc7-93c6-46e2-822d-13af89260e41 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:16.044913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-93bb3fc7-93c6-46e2-822d-13af89260e41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:16.044913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:16.044913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:16.044913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:16.044913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:16.044913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:16.044913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:16.044913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:16.044913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:16.044913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:16.044913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:16.044913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:16.044913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:16.044913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:16.044913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:16.046289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:16.047689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:16.047689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:16.047689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:16.047689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:16.047689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:16.047689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:16.047689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:16.047689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:16.055381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 359/711] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:16.056350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb3fc7-93c6-46e2-822d-13af89260e41 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:16.155013 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6c47f-f73c-4bad-bf45-ee8345cd3b2a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:16.260416 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cfd0e-9822-4500-991d-259fc5ce1a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:16.262596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cfd0e-9822-4500-991d-259fc5ce1a36 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:16.266098 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cfd0e-9822-4500-991d-259fc5ce1a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:16.266727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cfd0e-9822-4500-991d-259fc5ce1a36 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:16.268982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180cfd0e-9822-4500-991d-259fc5ce1a36 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:16.269868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180cfd0e-9822-4500-991d-259fc5ce1a36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:16.272402 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180cfd0e-9822-4500-991d-259fc5ce1a36 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:16.285069 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 353/712] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Jul 20 09:00:15 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:16.321993 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb3fc7-93c6-46e2-822d-13af89260e41 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:16.385506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a222f-01bb-4965-97c4-ded6b4f21dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:16.386190 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a222f-01bb-4965-97c4-ded6b4f21dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:16.386472 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a222f-01bb-4965-97c4-ded6b4f21dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:16.386771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a222f-01bb-4965-97c4-ded6b4f21dd3 None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:16.387580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a222f-01bb-4965-97c4-ded6b4f21dd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:16.387892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a222f-01bb-4965-97c4-ded6b4f21dd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:16.389946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a222f-01bb-4965-97c4-ded6b4f21dd3 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:16.413036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 360/713] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:16.419084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cfd0e-9822-4500-991d-259fc5ce1a36 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:16.474922 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:16.558417 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4519d8c-979d-4d2b-9595-b70e2e4222a4 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:16.558738 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4519d8c-979d-4d2b-9595-b70e2e4222a4 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:16.559056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4519d8c-979d-4d2b-9595-b70e2e4222a4 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:16.561248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 361/714] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:16.561730 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cfd0e-9822-4500-991d-259fc5ce1a36 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:16.566356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa3790-3fa6-42c0-a3c2-f91d86dcc38d None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:16.566647 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa3790-3fa6-42c0-a3c2-f91d86dcc38d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:16.566947 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa3790-3fa6-42c0-a3c2-f91d86dcc38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:16.598526 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-01fa3790-3fa6-42c0-a3c2-f91d86dcc38d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:16.599750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-01fa3790-3fa6-42c0-a3c2-f91d86dcc38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:16.623244 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894061bf-c6d4-4b9a-aa82-025f49ffd55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:16.623868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894061bf-c6d4-4b9a-aa82-025f49ffd55d None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:16.624160 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894061bf-c6d4-4b9a-aa82-025f49ffd55d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:16.624433 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894061bf-c6d4-4b9a-aa82-025f49ffd55d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:16.625264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894061bf-c6d4-4b9a-aa82-025f49ffd55d None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=a498284274454d25b1b118a88996e522, user.id=be08c43f0d0c4a8194596e54500b44ed) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:16.625573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894061bf-c6d4-4b9a-aa82-025f49ffd55d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:16.627016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894061bf-c6d4-4b9a-aa82-025f49ffd55d None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:16.631052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:16.644977 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 354/715] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:00:16 2022] GET /identity/v3/role_assignments?user.id=be08c43f0d0c4a8194596e54500b44ed&scope.project.id=a498284274454d25b1b118a88996e522&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:16.696751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 362/716] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:16.730916 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894061bf-c6d4-4b9a-aa82-025f49ffd55d None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:16.862393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894061bf-c6d4-4b9a-aa82-025f49ffd55d None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:16.866351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:16.905300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523f8c8-4fe2-4c7a-a2a9-a127cc2c5988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:16.906104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523f8c8-4fe2-4c7a-a2a9-a127cc2c5988 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:16.906380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523f8c8-4fe2-4c7a-a2a9-a127cc2c5988 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:16.906650 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523f8c8-4fe2-4c7a-a2a9-a127cc2c5988 None admin] PATH_INFO: `/v3/services/network` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:16.907362 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c523f8c8-4fe2-4c7a-a2a9-a127cc2c5988 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:16.908720 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c523f8c8-4fe2-4c7a-a2a9-a127cc2c5988 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-c523f8c8-4fe2-4c7a-a2a9-a127cc2c5988 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:16.928105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:16.929218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:16.929218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:16.929218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:16.929218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:16.929218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:16.929218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:16.929218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:16.929218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:16.929218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:16.929218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:16.929218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:16.929218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:16.929218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:16.929218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:16.929218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:16.929218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:16.930759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:16.930759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:16.930759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:16.930759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:16.930759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:16.930759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:16.930759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:16.930759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:16.930759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:16.930759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:16.930759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:16.930759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:16.930759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:00:16.930759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:16.930759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 355/717] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:16 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:16.935527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa3790-3fa6-42c0-a3c2-f91d86dcc38d None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:17.043163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa3790-3fa6-42c0-a3c2-f91d86dcc38d None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:17.086066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebab15-67b5-4a29-bb42-f83274ee1135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:17.086753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebab15-67b5-4a29-bb42-f83274ee1135 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:17.087041 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebab15-67b5-4a29-bb42-f83274ee1135 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:17.087319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebab15-67b5-4a29-bb42-f83274ee1135 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:17.088032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ebab15-67b5-4a29-bb42-f83274ee1135 None admin] RBAC: Adding query filter params (name=network) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:17.088337 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ebab15-67b5-4a29-bb42-f83274ee1135 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:17.089698 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ebab15-67b5-4a29-bb42-f83274ee1135 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:17.095278 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 363/718] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jul 20 09:00:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:17.100469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523f8c8-4fe2-4c7a-a2a9-a127cc2c5988 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:17.269530 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523f8c8-4fe2-4c7a-a2a9-a127cc2c5988 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:17.332385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5103c-d688-46fe-9d29-8e17e5e1152f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:17.332984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5103c-d688-46fe-9d29-8e17e5e1152f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:17.333269 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5103c-d688-46fe-9d29-8e17e5e1152f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:17.333547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5103c-d688-46fe-9d29-8e17e5e1152f None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:17.334269 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5103c-d688-46fe-9d29-8e17e5e1152f None admin] RBAC: Adding query filter params (type=network) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:17.334579 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5103c-d688-46fe-9d29-8e17e5e1152f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:17.335992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5103c-d688-46fe-9d29-8e17e5e1152f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:17.343951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 356/719] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jul 20 09:00:17 2022] GET /identity/v3/services?type=network => generated 124 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:17.714201 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ca479-1832-4c22-9322-3bd0b37bbf7a None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:17.714743 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ca479-1832-4c22-9322-3bd0b37bbf7a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:17.714823 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ca479-1832-4c22-9322-3bd0b37bbf7a None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:17.716395 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 364/720] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:17.721875 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223388f3-eb5f-4732-9a72-7223325f3045 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:17.722320 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223388f3-eb5f-4732-9a72-7223325f3045 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:17.722494 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223388f3-eb5f-4732-9a72-7223325f3045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:17.745741 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-223388f3-eb5f-4732-9a72-7223325f3045 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:17.746971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-223388f3-eb5f-4732-9a72-7223325f3045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:17.775962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:17.823052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 357/721] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:17.847734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366da1ce-ddd9-4756-b2bb-e5c26c74192e None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:17.848020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366da1ce-ddd9-4756-b2bb-e5c26c74192e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:17.848302 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366da1ce-ddd9-4756-b2bb-e5c26c74192e None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:17.849894 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 365/722] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:17.854971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a8e5b-126e-4d71-a8f0-bf557593dd59 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:17.855380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a8e5b-126e-4d71-a8f0-bf557593dd59 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:17.855773 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a8e5b-126e-4d71-a8f0-bf557593dd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:17.864398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366da1ce-ddd9-4756-b2bb-e5c26c74192e None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:17.871146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:17.878364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-c61a8e5b-126e-4d71-a8f0-bf557593dd59 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:17.881340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c61a8e5b-126e-4d71-a8f0-bf557593dd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:17.959494 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 358/723] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:17.990583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a8e5b-126e-4d71-a8f0-bf557593dd59 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:18.065694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366da1ce-ddd9-4756-b2bb-e5c26c74192e None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:18.136296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e38ab-08ae-4104-aa4f-fced88644ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:18.138222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e38ab-08ae-4104-aa4f-fced88644ace None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:18.143532 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e38ab-08ae-4104-aa4f-fced88644ace None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:18.144044 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e38ab-08ae-4104-aa4f-fced88644ace None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:18.163266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e38ab-08ae-4104-aa4f-fced88644ace None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:18.180769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-023e38ab-08ae-4104-aa4f-fced88644ace None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:00:18.180769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:18.180769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:18.180769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:18.180769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:18.180769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:18.180769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:18.180769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:18.180769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:18.180769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:18.180769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:18.180769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:18.180769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:18.180769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:18.180769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:18.180769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:18.181748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:18.181748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:18.181748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:18.181748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:18.181748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:18.181748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:18.181748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:18.181748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:18.181748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:18.181748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:18.181748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:18.181748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:18.181748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:18.181748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:18.181748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:18.181748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:18.182599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:18.182599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:18.182599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:18.182599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:18.182599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:18.182599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:18.182599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:18.182599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:18.182599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:18.182599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:18.182599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:00:18.182599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:18.186772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:18.187762 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 366/724] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:00:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:18.193948 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e38ab-08ae-4104-aa4f-fced88644ace None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:18.198044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a8e5b-126e-4d71-a8f0-bf557593dd59 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:18.257393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0362c-3a37-4050-b292-cc5046dbf459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:18.259091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0362c-3a37-4050-b292-cc5046dbf459 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:18.259379 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0362c-3a37-4050-b292-cc5046dbf459 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:18.259661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0362c-3a37-4050-b292-cc5046dbf459 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:18.283901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad0362c-3a37-4050-b292-cc5046dbf459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:18.292660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-3ad0362c-3a37-4050-b292-cc5046dbf459 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:18.292660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:18.292660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:18.292660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:18.292660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:18.292660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:18.292660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:18.292660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:18.292660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:18.292660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:18.292660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:18.292660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:18.292660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:18.292660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:18.292660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:18.292660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:18.293648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:18.293648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:18.293648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:18.293648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:18.293648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:18.293648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:18.293648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:18.293648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:18.293648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:18.293648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:18.293648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:18.293648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:18.293648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:18.293648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:18.293648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:18.293648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:18.295727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:18.295727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:18.295727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:18.295727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:18.295727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:18.295727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:18.295727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:18.295727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:18.295727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:18.295727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:18.295727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:18.295727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:18.295727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 359/725] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:00:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:18.300655 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0362c-3a37-4050-b292-cc5046dbf459 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:18.329312 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:18.405763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e38ab-08ae-4104-aa4f-fced88644ace None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:18.441934 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e00ea-3725-4228-8374-a89b7f88a3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:18.442608 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e00ea-3725-4228-8374-a89b7f88a3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:18.443024 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e00ea-3725-4228-8374-a89b7f88a3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:18.443308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e00ea-3725-4228-8374-a89b7f88a3f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:18.444062 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e00ea-3725-4228-8374-a89b7f88a3f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:18.444632 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e00ea-3725-4228-8374-a89b7f88a3f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:18.446005 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e00ea-3725-4228-8374-a89b7f88a3f7 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:18.456392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 367/726] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:00:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:18.459823 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0362c-3a37-4050-b292-cc5046dbf459 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:18.461862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e00ea-3725-4228-8374-a89b7f88a3f7 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:18.523555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3ab45-513c-4b2f-bfca-842d775f6759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:18.524149 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3ab45-513c-4b2f-bfca-842d775f6759 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:18.524432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3ab45-513c-4b2f-bfca-842d775f6759 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:18.524711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3ab45-513c-4b2f-bfca-842d775f6759 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:18.525465 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3ab45-513c-4b2f-bfca-842d775f6759 None admin] RBAC: Adding query filter params (name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:18.525775 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3ab45-513c-4b2f-bfca-842d775f6759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:18.530456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3ab45-513c-4b2f-bfca-842d775f6759 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:18.542267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 360/727] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 09:00:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:18.547988 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3ab45-513c-4b2f-bfca-842d775f6759 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:18.580177 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:18.641168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e00ea-3725-4228-8374-a89b7f88a3f7 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:18.678150 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:18.689426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c1f7a-08df-44af-acff-63a6112ea32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:18.690236 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c1f7a-08df-44af-acff-63a6112ea32b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:18.690519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c1f7a-08df-44af-acff-63a6112ea32b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:18.690827 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c1f7a-08df-44af-acff-63a6112ea32b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:18.703259 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c1f7a-08df-44af-acff-63a6112ea32b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:18.704874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c1f7a-08df-44af-acff-63a6112ea32b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:18.723075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-8d6c1f7a-08df-44af-acff-63a6112ea32b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:18.723075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:18.723075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:18.723075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:18.723075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:18.723075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:18.723075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:18.723075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:18.723075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:18.723075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:18.723075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:18.723075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:18.723075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:18.723075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:18.723075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:18.724420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:18.725311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:18.725311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:18.725311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:18.725311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:18.725311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 368/728] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:18.730197 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c1f7a-08df-44af-acff-63a6112ea32b None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:18.765610 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3ab45-513c-4b2f-bfca-842d775f6759 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:18.840380 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c1f7a-08df-44af-acff-63a6112ea32b None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:18.859403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98539832-ba66-4ba7-b6ab-9f3fffee30ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:18.860591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98539832-ba66-4ba7-b6ab-9f3fffee30ac None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:18.861071 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98539832-ba66-4ba7-b6ab-9f3fffee30ac None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:18.861560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98539832-ba66-4ba7-b6ab-9f3fffee30ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:18.876837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98539832-ba66-4ba7-b6ab-9f3fffee30ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:18.877044 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de96b7-dee5-466f-83e4-e2430fa0ff7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:18.877715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de96b7-dee5-466f-83e4-e2430fa0ff7d None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:18.878003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de96b7-dee5-466f-83e4-e2430fa0ff7d None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:18.878281 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de96b7-dee5-466f-83e4-e2430fa0ff7d None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:18.879042 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de96b7-dee5-466f-83e4-e2430fa0ff7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:18.879352 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de96b7-dee5-466f-83e4-e2430fa0ff7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:18.879790 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98539832-ba66-4ba7-b6ab-9f3fffee30ac None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:18.880731 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de96b7-dee5-466f-83e4-e2430fa0ff7d None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:18.894159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 369/729] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:18.897764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-98539832-ba66-4ba7-b6ab-9f3fffee30ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:18.897764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:18.897764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:18.897764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:18.897764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:18.897764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:18.897764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:18.897764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:18.897764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:18.897764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:18.897764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:18.897764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:18.897764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:18.897764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:18.897764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:18.898627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:18.899547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:18.899547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:18.899547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:18.899547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:18.899765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b57358-1ee5-4a01-a2e4-ea0830acccea None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:18.900053 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b57358-1ee5-4a01-a2e4-ea0830acccea None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:18.900508 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b57358-1ee5-4a01-a2e4-ea0830acccea None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:18.901843 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 361/730] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:18.902372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 370/731] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:18.902869 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98539832-ba66-4ba7-b6ab-9f3fffee30ac None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:18.908052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b57358-1ee5-4a01-a2e4-ea0830acccea None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:18.953804 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:18.979611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:19.017806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98539832-ba66-4ba7-b6ab-9f3fffee30ac None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:19.040274 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b57358-1ee5-4a01-a2e4-ea0830acccea None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:19.053595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb0920-4284-4975-a197-034aaa29e165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:19.054616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb0920-4284-4975-a197-034aaa29e165 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:19.055073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb0920-4284-4975-a197-034aaa29e165 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:19.055537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb0920-4284-4975-a197-034aaa29e165 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:19.064050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb0920-4284-4975-a197-034aaa29e165 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:19.065688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb0920-4284-4975-a197-034aaa29e165 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:19.081041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-6adb0920-4284-4975-a197-034aaa29e165 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:00:19.081041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:19.081041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:19.081041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:19.081041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:19.081041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:19.081041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:19.081041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:19.081041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:19.081041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:19.081041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:19.081041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:19.081041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:19.081041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:19.081041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:19.081041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:19.082033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:19.083018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:19.083018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:19.083018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:19.083018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:19.083018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:19.083018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:19.083018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:19.083018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:19.083018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:00:19.083018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:19.083551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51a198-0624-4c1f-af53-f265f9bc33a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:19.083551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51a198-0624-4c1f-af53-f265f9bc33a5 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:19.083551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51a198-0624-4c1f-af53-f265f9bc33a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:19.083786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51a198-0624-4c1f-af53-f265f9bc33a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:19.084721 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e51a198-0624-4c1f-af53-f265f9bc33a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:19.085183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e51a198-0624-4c1f-af53-f265f9bc33a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:19.086068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 362/732] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:00:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:19.086950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e51a198-0624-4c1f-af53-f265f9bc33a5 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:19.087785 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5dba8-6633-4fb7-8cf8-8e4006220146 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:19.088175 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5dba8-6633-4fb7-8cf8-8e4006220146 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:19.088569 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5dba8-6633-4fb7-8cf8-8e4006220146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:19.102577 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 371/733] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:19.103650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51a198-0624-4c1f-af53-f265f9bc33a5 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:19.106090 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-66a5dba8-6633-4fb7-8cf8-8e4006220146 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:19.108323 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-66a5dba8-6633-4fb7-8cf8-8e4006220146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:19.190814 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 363/734] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:19.192027 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5dba8-6633-4fb7-8cf8-8e4006220146 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:19.208450 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:19.212320 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51a198-0624-4c1f-af53-f265f9bc33a5 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:19.248952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfd9f0-2385-455b-a54e-cae141571b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:19.249631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfd9f0-2385-455b-a54e-cae141571b08 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:19.249920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfd9f0-2385-455b-a54e-cae141571b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:19.250204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfd9f0-2385-455b-a54e-cae141571b08 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:19.251074 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfd9f0-2385-455b-a54e-cae141571b08 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:19.251397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfd9f0-2385-455b-a54e-cae141571b08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:19.252835 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfd9f0-2385-455b-a54e-cae141571b08 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:19.256143 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:19.276703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 372/735] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Jul 20 09:00:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:19.276703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f736474-dd4e-4f7c-bda5-af3b6a272e9d None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:19.276703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f736474-dd4e-4f7c-bda5-af3b6a272e9d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:19.276703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f736474-dd4e-4f7c-bda5-af3b6a272e9d None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:19.276703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 373/736] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:19.277142 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f736474-dd4e-4f7c-bda5-af3b6a272e9d None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:19.311056 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5dba8-6633-4fb7-8cf8-8e4006220146 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:19.348324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ceac55-c088-4735-8bf8-3f700d219466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:19.349153 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ceac55-c088-4735-8bf8-3f700d219466 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:19.349440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ceac55-c088-4735-8bf8-3f700d219466 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:19.349719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ceac55-c088-4735-8bf8-3f700d219466 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:19.358964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ceac55-c088-4735-8bf8-3f700d219466 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:19.360674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ceac55-c088-4735-8bf8-3f700d219466 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:19.375967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-d6ceac55-c088-4735-8bf8-3f700d219466 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:00:19.375967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:19.375967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:19.375967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:19.375967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:19.375967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:19.375967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:19.375967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:19.375967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:19.375967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:19.375967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:19.375967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:19.375967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:19.375967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:19.375967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:19.375967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:19.376953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:19.377905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:19.377905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:19.377905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:19.377905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:19.377905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:19.377905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:19.377905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:19.377905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:19.377905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:00:19.377905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:19.377905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 364/737] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jul 20 09:00:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:19.378496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ceac55-c088-4735-8bf8-3f700d219466 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:19.387715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f736474-dd4e-4f7c-bda5-af3b6a272e9d None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:19.428197 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b486b-d342-4c2b-853b-05ebddcc98d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:19.429135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b486b-d342-4c2b-853b-05ebddcc98d5 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:19.429425 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b486b-d342-4c2b-853b-05ebddcc98d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:19.429902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b486b-d342-4c2b-853b-05ebddcc98d5 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:19.430945 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b486b-d342-4c2b-853b-05ebddcc98d5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:19.432355 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b486b-d342-4c2b-853b-05ebddcc98d5 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:19.450210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 374/738] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jul 20 09:00:19 2022] POST /identity/v3/services => generated 251 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:00:19.452148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c57fb1-1795-45d1-9885-40a85b1299ea None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:19.452503 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c57fb1-1795-45d1-9885-40a85b1299ea None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:19.452855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c57fb1-1795-45d1-9885-40a85b1299ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:19.482718 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-f9c57fb1-1795-45d1-9885-40a85b1299ea None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:19.482718 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f9c57fb1-1795-45d1-9885-40a85b1299ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:19.484542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:19.523101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ceac55-c088-4735-8bf8-3f700d219466 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:19.559690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:19.572081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 375/739] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:19.573311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c57fb1-1795-45d1-9885-40a85b1299ea None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:19.600980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df1931-db74-4fc6-9769-99aad6d3493e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:19.602006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df1931-db74-4fc6-9769-99aad6d3493e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:19.602519 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df1931-db74-4fc6-9769-99aad6d3493e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:19.603018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df1931-db74-4fc6-9769-99aad6d3493e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:19.684668 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df1931-db74-4fc6-9769-99aad6d3493e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:19.686663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df1931-db74-4fc6-9769-99aad6d3493e None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:19.718072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c57fb1-1795-45d1-9885-40a85b1299ea None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:19.756443 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df766797-ba85-4f48-a53a-9de115888e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:19.757200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df766797-ba85-4f48-a53a-9de115888e32 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:19.757394 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df766797-ba85-4f48-a53a-9de115888e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:19.757683 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df766797-ba85-4f48-a53a-9de115888e32 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:19.758634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df766797-ba85-4f48-a53a-9de115888e32 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:19.758985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df766797-ba85-4f48-a53a-9de115888e32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:19.760668 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df766797-ba85-4f48-a53a-9de115888e32 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:19.780738 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 376/740] 10.222.0.44 () {56 vars in 1149 bytes} [Wed Jul 20 09:00:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:19.781604 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df766797-ba85-4f48-a53a-9de115888e32 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:19.785351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:19.802340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 365/741] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:00:19 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:00:19.803525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df1931-db74-4fc6-9769-99aad6d3493e None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:19.827643 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:19.921487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df1931-db74-4fc6-9769-99aad6d3493e None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:19.925157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df766797-ba85-4f48-a53a-9de115888e32 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:19.958349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f7577-6c92-446d-875d-868e47eb9167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:19.959037 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f7577-6c92-446d-875d-868e47eb9167 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:19.959316 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f7577-6c92-446d-875d-868e47eb9167 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:19.959700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f7577-6c92-446d-875d-868e47eb9167 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:19.967178 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb609830-01a6-4ec3-8511-6452db83d61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:19.967977 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb609830-01a6-4ec3-8511-6452db83d61e None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:19.968300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb609830-01a6-4ec3-8511-6452db83d61e None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:19.968544 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb609830-01a6-4ec3-8511-6452db83d61e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:19.979995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb609830-01a6-4ec3-8511-6452db83d61e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:19.988628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-bb609830-01a6-4ec3-8511-6452db83d61e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:00:19.988628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:19.988628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:19.988628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:19.988628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:19.988628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:19.988628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:19.988628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:19.988628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:19.988628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:19.988628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:19.988628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:19.988628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:19.988628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:19.988628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:19.988628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:19.989573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:19.989573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:19.989573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:19.989573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:19.989573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:19.989573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:19.989573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:19.989573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:19.989573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:19.989573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:19.989573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:19.989573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:19.989573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:19.989573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:19.989573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:19.989573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:19.990406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:19.990406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:19.990406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:19.990406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:19.990406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:19.990406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:19.990406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:19.990406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:19.990406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:19.990406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:19.990406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:00:19.990406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:19.990406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 377/742] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:00:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:19.991141 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb609830-01a6-4ec3-8511-6452db83d61e None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:20.037501 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f7577-6c92-446d-875d-868e47eb9167 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:20.039309 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f7577-6c92-446d-875d-868e47eb9167 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:20.055925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:20.056586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:20.101694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb609830-01a6-4ec3-8511-6452db83d61e None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:20.140313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de1e18-6603-4d36-b783-a86dba9ad753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:20.141400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de1e18-6603-4d36-b783-a86dba9ad753 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:20.141854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de1e18-6603-4d36-b783-a86dba9ad753 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:20.142126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de1e18-6603-4d36-b783-a86dba9ad753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:20.157979 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 366/743] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:00:19 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 09:00:20.158204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de1e18-6603-4d36-b783-a86dba9ad753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:20.158311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94307d1c-ad1e-4994-8752-23b8fe8b304f None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:20.158388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de1e18-6603-4d36-b783-a86dba9ad753 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:20.158460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94307d1c-ad1e-4994-8752-23b8fe8b304f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:20.158460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94307d1c-ad1e-4994-8752-23b8fe8b304f None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:20.158460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 367/744] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:20.158460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94307d1c-ad1e-4994-8752-23b8fe8b304f None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:20.177739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-13de1e18-6603-4d36-b783-a86dba9ad753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:20.177739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:20.177739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:20.177739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:20.177739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:20.177739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:20.177739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:20.177739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:20.177739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:20.177739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:20.177739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:20.177739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:20.177739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:20.177739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:20.177739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:20.178749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:20.180319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:20.180319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:20.180319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:20.180319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:20.180319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 378/745] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:20.180319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de1e18-6603-4d36-b783-a86dba9ad753 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:20.271162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94307d1c-ad1e-4994-8752-23b8fe8b304f None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:20.286766 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de1e18-6603-4d36-b783-a86dba9ad753 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:20.289820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:20.300989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:20.312421 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb93a4c-1117-42c3-857c-0c5a61605b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:20.313324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93a4c-1117-42c3-857c-0c5a61605b93 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:20.313763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93a4c-1117-42c3-857c-0c5a61605b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:20.314203 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb93a4c-1117-42c3-857c-0c5a61605b93 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:20.315370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb93a4c-1117-42c3-857c-0c5a61605b93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:20.315684 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb93a4c-1117-42c3-857c-0c5a61605b93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:20.317043 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb93a4c-1117-42c3-857c-0c5a61605b93 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:20.323090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47250082-1b68-4845-906a-6004e0dd3236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:20.323874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47250082-1b68-4845-906a-6004e0dd3236 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:20.324153 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47250082-1b68-4845-906a-6004e0dd3236 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:20.324425 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47250082-1b68-4845-906a-6004e0dd3236 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:20.332628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 368/746] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:00:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:20.332628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f2a00-05af-4ed0-9d3d-9d18650efda9 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:20.332628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f2a00-05af-4ed0-9d3d-9d18650efda9 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:20.332628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f2a00-05af-4ed0-9d3d-9d18650efda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:20.336833 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47250082-1b68-4845-906a-6004e0dd3236 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:20.338284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47250082-1b68-4845-906a-6004e0dd3236 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:20.346800 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-688f2a00-05af-4ed0-9d3d-9d18650efda9 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:20.348012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-688f2a00-05af-4ed0-9d3d-9d18650efda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:20.355187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-47250082-1b68-4845-906a-6004e0dd3236 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:20.355187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:20.355187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:20.355187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:20.355187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:20.355187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:20.355187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:20.355187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:20.355187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:20.355187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:20.355187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:20.355187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:20.355187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:20.355187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:20.355187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:20.356095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:20.357039 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:20.357039 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:20.357039 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:20.357039 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:20.357039 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 379/747] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:20.357524 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47250082-1b68-4845-906a-6004e0dd3236 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:20.427397 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 369/748] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 09:00:20.428530 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f2a00-05af-4ed0-9d3d-9d18650efda9 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:20.481705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47250082-1b68-4845-906a-6004e0dd3236 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:20.524996 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863dd16-684f-48a3-8dc2-6d2455626546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:20.525615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863dd16-684f-48a3-8dc2-6d2455626546 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:20.525903 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863dd16-684f-48a3-8dc2-6d2455626546 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:20.526180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863dd16-684f-48a3-8dc2-6d2455626546 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:20.526928 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863dd16-684f-48a3-8dc2-6d2455626546 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:20.527234 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863dd16-684f-48a3-8dc2-6d2455626546 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:20.529314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863dd16-684f-48a3-8dc2-6d2455626546 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:20.532868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f2a00-05af-4ed0-9d3d-9d18650efda9 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:20.542273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 380/749] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:20.543801 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863dd16-684f-48a3-8dc2-6d2455626546 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:20.548228 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:20.575106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ad570-64f9-4ad7-91a3-20bb05b840ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:20.576142 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ad570-64f9-4ad7-91a3-20bb05b840ea None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:20.576428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ad570-64f9-4ad7-91a3-20bb05b840ea None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:20.576706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ad570-64f9-4ad7-91a3-20bb05b840ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:20.578822 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:20.586884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ad570-64f9-4ad7-91a3-20bb05b840ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:20.588322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ad570-64f9-4ad7-91a3-20bb05b840ea None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:20.606130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-187ad570-64f9-4ad7-91a3-20bb05b840ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:20.606130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:20.606130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:20.606130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:20.606130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:20.606130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:20.606130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:20.606130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:20.606130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:20.606130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:20.606130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:20.606130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:20.606130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:20.606130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:20.606130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:20.607850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:20.614091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:20.614091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:20.614091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:20.614091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:20.614091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 370/750] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:20.614091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ad570-64f9-4ad7-91a3-20bb05b840ea None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:20.727510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ad570-64f9-4ad7-91a3-20bb05b840ea None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:20.771703 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff401eac-3d25-4e77-b5c2-4771c053825d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:20.773044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff401eac-3d25-4e77-b5c2-4771c053825d None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:20.773491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff401eac-3d25-4e77-b5c2-4771c053825d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:20.774165 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff401eac-3d25-4e77-b5c2-4771c053825d None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:20.775292 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff401eac-3d25-4e77-b5c2-4771c053825d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:20.777445 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff401eac-3d25-4e77-b5c2-4771c053825d None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:20.781866 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863dd16-684f-48a3-8dc2-6d2455626546 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:20.795497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 371/751] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jul 20 09:00:20 2022] POST /identity/v3/services => generated 240 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:00:20.808496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff401eac-3d25-4e77-b5c2-4771c053825d None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:20.826700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f38dd-d592-4c01-b706-7e91defdf5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:20.827669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f38dd-d592-4c01-b706-7e91defdf5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:20.827996 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f38dd-d592-4c01-b706-7e91defdf5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:20.828813 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f38dd-d592-4c01-b706-7e91defdf5b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:20.829136 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f38dd-d592-4c01-b706-7e91defdf5b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:20.829444 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f38dd-d592-4c01-b706-7e91defdf5b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:20.830879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f38dd-d592-4c01-b706-7e91defdf5b4 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:20.845151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:20.849283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 381/752] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:20.850281 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f38dd-d592-4c01-b706-7e91defdf5b4 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:20.952867 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:20.959688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff401eac-3d25-4e77-b5c2-4771c053825d None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:20.979260 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f38dd-d592-4c01-b706-7e91defdf5b4 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:20.999098 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb1643-48c0-4ad9-8c95-fe5fc54e36b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:21.000020 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb1643-48c0-4ad9-8c95-fe5fc54e36b2 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:21.000331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb1643-48c0-4ad9-8c95-fe5fc54e36b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:21.000684 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb1643-48c0-4ad9-8c95-fe5fc54e36b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:21.008264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb1643-48c0-4ad9-8c95-fe5fc54e36b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:21.009681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb1643-48c0-4ad9-8c95-fe5fc54e36b2 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:21.020756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24400b6-930b-4e39-b2da-06e92d28e7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:21.021433 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24400b6-930b-4e39-b2da-06e92d28e7cd None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:21.021724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24400b6-930b-4e39-b2da-06e92d28e7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:21.022005 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24400b6-930b-4e39-b2da-06e92d28e7cd None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:21.023220 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24400b6-930b-4e39-b2da-06e92d28e7cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:21.023220 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24400b6-930b-4e39-b2da-06e92d28e7cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:21.024460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24400b6-930b-4e39-b2da-06e92d28e7cd None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:21.024781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-0fcb1643-48c0-4ad9-8c95-fe5fc54e36b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:21.024781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:21.024781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:21.024781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:21.024781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:21.024781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:21.024781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:21.024781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:21.024781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:21.024781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:21.024781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:21.024781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:21.024781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:21.024781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:21.024781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:21.025688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:21.026595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:21.026595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:21.026595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:21.026595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:21.026595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:21.026595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:21.026595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:21.026595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:21.026595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 372/753] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:21.027519 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb1643-48c0-4ad9-8c95-fe5fc54e36b2 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:21.041527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 382/754] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:21.042722 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24400b6-930b-4e39-b2da-06e92d28e7cd None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:21.127373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:21.153016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb1643-48c0-4ad9-8c95-fe5fc54e36b2 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:21.176655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24400b6-930b-4e39-b2da-06e92d28e7cd None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:21.197083 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97e4a8-03a9-4813-aab4-1b2ab08bfaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:21.198186 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97e4a8-03a9-4813-aab4-1b2ab08bfaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:21.198483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97e4a8-03a9-4813-aab4-1b2ab08bfaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:21.198796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97e4a8-03a9-4813-aab4-1b2ab08bfaf0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:21.208339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97e4a8-03a9-4813-aab4-1b2ab08bfaf0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:21.210253 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97e4a8-03a9-4813-aab4-1b2ab08bfaf0 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:21.215844 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd87eb-f766-4ae7-bc50-16953391f0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:21.217043 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd87eb-f766-4ae7-bc50-16953391f0cd None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:21.217604 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd87eb-f766-4ae7-bc50-16953391f0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:21.217881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd87eb-f766-4ae7-bc50-16953391f0cd None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:21.218689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd87eb-f766-4ae7-bc50-16953391f0cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:21.219022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd87eb-f766-4ae7-bc50-16953391f0cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:21.220451 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd87eb-f766-4ae7-bc50-16953391f0cd None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:21.226114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-ae97e4a8-03a9-4813-aab4-1b2ab08bfaf0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:21.226114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:21.226114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:21.226114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:21.226114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:21.226114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:21.226114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:21.226114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:21.226114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:21.226114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:21.226114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:21.226114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:21.226114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:21.226114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:21.226114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:21.227060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:21.228014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:21.228014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:21.228014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:21.228014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:21.228014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:21.228014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:21.228014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:21.228014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:21.228441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 373/755] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:21.228951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97e4a8-03a9-4813-aab4-1b2ab08bfaf0 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:21.234471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:21.248398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 383/756] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:21.248398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd87eb-f766-4ae7-bc50-16953391f0cd None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:21.355724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97e4a8-03a9-4813-aab4-1b2ab08bfaf0 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:21.364633 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd87eb-f766-4ae7-bc50-16953391f0cd None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:21.393509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c4ad0-c1aa-4c6e-8073-b9b5ada19567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:21.394303 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c4ad0-c1aa-4c6e-8073-b9b5ada19567 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:21.394632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c4ad0-c1aa-4c6e-8073-b9b5ada19567 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:21.394940 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c4ad0-c1aa-4c6e-8073-b9b5ada19567 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:21.403518 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006362a-6f83-4664-b6cb-ce7859f132d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:21.403740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c4ad0-c1aa-4c6e-8073-b9b5ada19567 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:21.404815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006362a-6f83-4664-b6cb-ce7859f132d7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:21.405195 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c4ad0-c1aa-4c6e-8073-b9b5ada19567 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:21.405533 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006362a-6f83-4664-b6cb-ce7859f132d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:21.405980 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006362a-6f83-4664-b6cb-ce7859f132d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:21.406973 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006362a-6f83-4664-b6cb-ce7859f132d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:21.407480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006362a-6f83-4664-b6cb-ce7859f132d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:21.408184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:21.409873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006362a-6f83-4664-b6cb-ce7859f132d7 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:21.422107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-f42c4ad0-c1aa-4c6e-8073-b9b5ada19567 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:00:21.422107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:21.422107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:21.422107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:21.422107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:21.422107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:21.422107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:21.422107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:21.422107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:21.422107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:21.422107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:21.422107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:21.422107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:21.422107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:21.422107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:21.422107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:21.423331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:21.424503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:21.424503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:21.424503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:21.424503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:21.424503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:21.424503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:21.424503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:21.424503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:21.424503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:00:21.424503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:21.426787 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 384/757] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:21.427783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006362a-6f83-4664-b6cb-ce7859f132d7 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:21.432463 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 374/758] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jul 20 09:00:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:21.437938 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c4ad0-c1aa-4c6e-8073-b9b5ada19567 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:21.527056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:21.540735 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006362a-6f83-4664-b6cb-ce7859f132d7 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:21.554251 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c4ad0-c1aa-4c6e-8073-b9b5ada19567 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:21.581486 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867f3ba-524e-4f70-8983-4ba363ae0484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:21.582383 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867f3ba-524e-4f70-8983-4ba363ae0484 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:21.582894 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867f3ba-524e-4f70-8983-4ba363ae0484 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:21.583325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867f3ba-524e-4f70-8983-4ba363ae0484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:21.584426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867f3ba-524e-4f70-8983-4ba363ae0484 None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=a498284274454d25b1b118a88996e522, user.id=03b4375e6ead42fb8d77a7b30617cb95) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:21.584952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867f3ba-524e-4f70-8983-4ba363ae0484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:21.586544 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867f3ba-524e-4f70-8983-4ba363ae0484 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:21.598937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a7b96-d769-4dea-a4fe-ee96506a919e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:21.598937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a7b96-d769-4dea-a4fe-ee96506a919e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:21.598937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a7b96-d769-4dea-a4fe-ee96506a919e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:21.598937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a7b96-d769-4dea-a4fe-ee96506a919e None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:21.599409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a7b96-d769-4dea-a4fe-ee96506a919e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:21.599409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a7b96-d769-4dea-a4fe-ee96506a919e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:21.599409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a7b96-d769-4dea-a4fe-ee96506a919e None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:21.599823 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 385/759] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:00:21 2022] GET /identity/v3/role_assignments?user.id=03b4375e6ead42fb8d77a7b30617cb95&scope.project.id=a498284274454d25b1b118a88996e522&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:21.601052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867f3ba-524e-4f70-8983-4ba363ae0484 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:21.624863 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 375/760] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jul 20 09:00:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:21.626528 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36915b5-a906-4688-bcb4-fe340e1a3b56 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:21.626996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36915b5-a906-4688-bcb4-fe340e1a3b56 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:21.627330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36915b5-a906-4688-bcb4-fe340e1a3b56 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:21.629693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 376/761] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:21.630847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36915b5-a906-4688-bcb4-fe340e1a3b56 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:21.719946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:21.726003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867f3ba-524e-4f70-8983-4ba363ae0484 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:21.753236 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:21.766394 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effcfec-3615-4bd3-8ce5-ece797cecab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:21.767473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effcfec-3615-4bd3-8ce5-ece797cecab7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:21.767995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effcfec-3615-4bd3-8ce5-ece797cecab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:21.768481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effcfec-3615-4bd3-8ce5-ece797cecab7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:21.769687 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effcfec-3615-4bd3-8ce5-ece797cecab7 None admin] RBAC: Adding query filter params (role.id=ae949d8bea4b40d9b1a9fa207d539663, scope.project.id=a498284274454d25b1b118a88996e522, user.id=6b8db4a7ee284b1aa96d4517dd381e71) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:21.770195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effcfec-3615-4bd3-8ce5-ece797cecab7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:21.771919 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effcfec-3615-4bd3-8ce5-ece797cecab7 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:21.779163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 386/762] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:00:21 2022] GET /identity/v3/role_assignments?user.id=6b8db4a7ee284b1aa96d4517dd381e71&scope.project.id=a498284274454d25b1b118a88996e522&role.id=ae949d8bea4b40d9b1a9fa207d539663 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:21.780703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adcc1d-1ae8-4413-870e-a3b490139c00 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:21.781129 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adcc1d-1ae8-4413-870e-a3b490139c00 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:21.781569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3adcc1d-1ae8-4413-870e-a3b490139c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:21.791658 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36915b5-a906-4688-bcb4-fe340e1a3b56 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:21.802615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-f3adcc1d-1ae8-4413-870e-a3b490139c00 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:21.804428 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f3adcc1d-1ae8-4413-870e-a3b490139c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:21.864514 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d3429-2256-49e1-bc08-3f561d189fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:21.865932 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d3429-2256-49e1-bc08-3f561d189fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:21.867227 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d3429-2256-49e1-bc08-3f561d189fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:21.867687 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d3429-2256-49e1-bc08-3f561d189fc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:21.888957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d3429-2256-49e1-bc08-3f561d189fc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:21.892302 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435d3429-2256-49e1-bc08-3f561d189fc4 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:21.920322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-435d3429-2256-49e1-bc08-3f561d189fc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:21.920322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:21.920322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:21.920322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:21.920322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:21.920322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:21.920322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:21.920322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:21.920322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:21.920322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:21.920322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:21.920322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:21.920322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:21.920322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:21.920322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:21.921460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:21.922551 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:21.922551 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:21.922551 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:21.922551 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:21.932652 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 377/763] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:21.936118 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 387/764] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:21.939560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d3429-2256-49e1-bc08-3f561d189fc4 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:21.962199 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3adcc1d-1ae8-4413-870e-a3b490139c00 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:22.096537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3adcc1d-1ae8-4413-870e-a3b490139c00 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:22.116249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:22.136654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ba13a-7c17-4532-89e6-ccc582be6218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:22.137994 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ba13a-7c17-4532-89e6-ccc582be6218 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:22.138744 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ba13a-7c17-4532-89e6-ccc582be6218 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:22.139309 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ba13a-7c17-4532-89e6-ccc582be6218 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:22.140191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ba13a-7c17-4532-89e6-ccc582be6218 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:22.141773 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ba13a-7c17-4532-89e6-ccc582be6218 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:22.154765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-ac9ba13a-7c17-4532-89e6-ccc582be6218 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:00:22.154765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:22.154765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:22.154765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:22.154765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:22.154765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:22.154765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:22.154765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:22.154765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:00:22.154765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:22.154765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:22.154765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:22.154765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:22.154765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:22.154765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:22.154765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:22.155755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:22.155755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:22.155755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:22.155755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:22.155755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:22.155755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:22.155755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:22.155755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:22.155755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:22.155755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:22.155755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:22.155755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:22.155755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:22.155755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:22.155755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:22.155755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:22.156602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:22.156602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:22.156602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:22.156602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:22.156602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:22.156602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:22.156602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:22.156602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:22.156602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:22.156602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:22.156602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:22.156602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:22.156602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:22.156602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:00:22.156602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:22.159638 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 388/765] 10.222.0.44 () {56 vars in 1129 bytes} [Wed Jul 20 09:00:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:22.166196 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ba13a-7c17-4532-89e6-ccc582be6218 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:22.172564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d3429-2256-49e1-bc08-3f561d189fc4 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:22.195062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:22.275456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b1cf1-d10f-4b4b-8f4c-9a1ce88774c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:22.276162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b1cf1-d10f-4b4b-8f4c-9a1ce88774c3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:22.276451 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b1cf1-d10f-4b4b-8f4c-9a1ce88774c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:22.276788 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b1cf1-d10f-4b4b-8f4c-9a1ce88774c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:22.277453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b1cf1-d10f-4b4b-8f4c-9a1ce88774c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:22.277758 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b1cf1-d10f-4b4b-8f4c-9a1ce88774c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:22.279776 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b1cf1-d10f-4b4b-8f4c-9a1ce88774c3 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:22.296209 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ba13a-7c17-4532-89e6-ccc582be6218 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:22.312967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 378/766] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:22.318300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b1cf1-d10f-4b4b-8f4c-9a1ce88774c3 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:22.345060 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89232a-680b-4ad4-83f4-0c0de95fd8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:22.345737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89232a-680b-4ad4-83f4-0c0de95fd8ca None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:22.346017 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89232a-680b-4ad4-83f4-0c0de95fd8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:22.346293 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89232a-680b-4ad4-83f4-0c0de95fd8ca None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:22.349058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89232a-680b-4ad4-83f4-0c0de95fd8ca None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:22.349371 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89232a-680b-4ad4-83f4-0c0de95fd8ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:22.350808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89232a-680b-4ad4-83f4-0c0de95fd8ca None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:22.362573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 389/767] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Jul 20 09:00:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:22.370401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89232a-680b-4ad4-83f4-0c0de95fd8ca None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:22.440739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:22.445358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b1cf1-d10f-4b4b-8f4c-9a1ce88774c3 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:22.496104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e678e9-bc73-48f4-8d3e-2be428e3cab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:22.497193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e678e9-bc73-48f4-8d3e-2be428e3cab3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:22.498132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e678e9-bc73-48f4-8d3e-2be428e3cab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:22.498553 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e678e9-bc73-48f4-8d3e-2be428e3cab3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:22.498770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89232a-680b-4ad4-83f4-0c0de95fd8ca None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:22.509440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e678e9-bc73-48f4-8d3e-2be428e3cab3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:22.516229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e678e9-bc73-48f4-8d3e-2be428e3cab3 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:22.543545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c37c6-12cc-4576-b015-ff7f12517bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:22.544962 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c37c6-12cc-4576-b015-ff7f12517bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:22.545352 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c37c6-12cc-4576-b015-ff7f12517bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:22.545734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c37c6-12cc-4576-b015-ff7f12517bb5 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:22.546568 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c37c6-12cc-4576-b015-ff7f12517bb5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:22.547016 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c37c6-12cc-4576-b015-ff7f12517bb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:22.548598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c37c6-12cc-4576-b015-ff7f12517bb5 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:22.551935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-74e678e9-bc73-48f4-8d3e-2be428e3cab3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:22.551935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:22.551935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:22.551935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:22.551935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:22.551935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:22.551935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:22.551935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:22.551935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:22.551935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:22.551935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:22.551935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:22.551935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:22.551935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:22.551935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:22.552946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:22.553832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:22.553832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:22.553832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:22.553832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:22.553832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:22.553832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:22.553832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:22.553832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:22.558651 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 379/768] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:22.560508 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e678e9-bc73-48f4-8d3e-2be428e3cab3 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:22.563660 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 390/769] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Jul 20 09:00:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:22.569536 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c37c6-12cc-4576-b015-ff7f12517bb5 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:22.620262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:22.677038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c37c6-12cc-4576-b015-ff7f12517bb5 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:22.688069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:22.717663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2a5d7-8376-4c50-be57-c9ebac12c55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:22.718662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2a5d7-8376-4c50-be57-c9ebac12c55a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:22.719181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2a5d7-8376-4c50-be57-c9ebac12c55a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:22.719670 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2a5d7-8376-4c50-be57-c9ebac12c55a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:22.720676 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2a5d7-8376-4c50-be57-c9ebac12c55a None admin] RBAC: Adding query filter params (interface=public, service_id=c5a35922a6fe4c73a38f841d5af32128, region_id=RegionOne) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:22.721188 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2a5d7-8376-4c50-be57-c9ebac12c55a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:22.723300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2a5d7-8376-4c50-be57-c9ebac12c55a None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:22.733500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 391/770] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Jul 20 09:00:22 2022] GET /identity/v3/endpoints?service_id=c5a35922a6fe4c73a38f841d5af32128&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:22.739160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2a5d7-8376-4c50-be57-c9ebac12c55a None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:22.786632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e678e9-bc73-48f4-8d3e-2be428e3cab3 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:22.852078 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ffe662-651a-4b24-bd82-19f530c561b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:22.852745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ffe662-651a-4b24-bd82-19f530c561b7 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:22.853031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ffe662-651a-4b24-bd82-19f530c561b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:22.853310 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ffe662-651a-4b24-bd82-19f530c561b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:22.854105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ffe662-651a-4b24-bd82-19f530c561b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:22.854419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ffe662-651a-4b24-bd82-19f530c561b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:22.855899 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ffe662-651a-4b24-bd82-19f530c561b7 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:22.877041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 380/771] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:22.883611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ffe662-651a-4b24-bd82-19f530c561b7 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:22.914018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2a5d7-8376-4c50-be57-c9ebac12c55a None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:22.974121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81993e49-861f-4362-8275-e82ea0a27eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:22.974740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81993e49-861f-4362-8275-e82ea0a27eab None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:22.975032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81993e49-861f-4362-8275-e82ea0a27eab None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:22.975311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81993e49-861f-4362-8275-e82ea0a27eab None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:22.975997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81993e49-861f-4362-8275-e82ea0a27eab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:22.977405 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81993e49-861f-4362-8275-e82ea0a27eab None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:22.986936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 392/772] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jul 20 09:00:22 2022] GET /identity/v3/services => generated 772 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:23.032699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:23.052349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ffe662-651a-4b24-bd82-19f530c561b7 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:23.066170 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13a87c-f428-4fb8-b64c-e3f2d809d47b None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:23.066631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13a87c-f428-4fb8-b64c-e3f2d809d47b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:23.067116 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13a87c-f428-4fb8-b64c-e3f2d809d47b None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:23.068881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 393/773] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:23.074451 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798690c7-677a-4d1e-8faa-da1e31c0fc16 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:23.074881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798690c7-677a-4d1e-8faa-da1e31c0fc16 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:23.075269 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798690c7-677a-4d1e-8faa-da1e31c0fc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:23.102181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a324dd2-8cbb-49ec-a2ac-bbdbd973edc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:23.103385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a324dd2-8cbb-49ec-a2ac-bbdbd973edc9 None admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:23.103812 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a324dd2-8cbb-49ec-a2ac-bbdbd973edc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:23.104482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a324dd2-8cbb-49ec-a2ac-bbdbd973edc9 None admin] PATH_INFO: `/v3/projects/a498284274454d25b1b118a88996e522/users/be08c43f0d0c4a8194596e54500b44ed/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:23.110596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a324dd2-8cbb-49ec-a2ac-bbdbd973edc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a498284274454d25b1b118a88996e522, user_id=be08c43f0d0c4a8194596e54500b44ed, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:23.112334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a324dd2-8cbb-49ec-a2ac-bbdbd973edc9 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:23.123384 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-798690c7-677a-4d1e-8faa-da1e31c0fc16 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:23.125226 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-798690c7-677a-4d1e-8faa-da1e31c0fc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:23.138897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 381/774] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 09:00:22 2022] PUT /identity/v3/projects/a498284274454d25b1b118a88996e522/users/be08c43f0d0c4a8194596e54500b44ed/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:00:23.196095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:23.245880 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 394/775] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:23.270937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a324dd2-8cbb-49ec-a2ac-bbdbd973edc9 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:23.383201 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a324dd2-8cbb-49ec-a2ac-bbdbd973edc9 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:23.428549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea5cc1-0d40-45cc-8526-82af6749bd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:23.429350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea5cc1-0d40-45cc-8526-82af6749bd58 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:23.429631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea5cc1-0d40-45cc-8526-82af6749bd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:23.429904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea5cc1-0d40-45cc-8526-82af6749bd58 None admin] PATH_INFO: `/v3/services/network` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:23.430556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea5cc1-0d40-45cc-8526-82af6749bd58 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:23.431932 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea5cc1-0d40-45cc-8526-82af6749bd58 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-1bea5cc1-0d40-45cc-8526-82af6749bd58 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:23.447189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:23.448184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:23.448184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:23.448184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:23.448184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:23.448184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:23.448184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:23.448184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:23.448184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:23.448184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:23.448184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:23.448184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:23.448184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:23.448184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:23.448184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:23.448184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:23.448184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:23.449018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:23.449018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:23.449018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:23.449018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:23.449018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:23.449018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:23.449018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:23.449018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:23.449018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:23.449018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:23.449018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:23.449018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:23.449018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:00:23.449018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:23.453096 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 382/776] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:23 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:23.454389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798690c7-677a-4d1e-8faa-da1e31c0fc16 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:23.616184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798690c7-677a-4d1e-8faa-da1e31c0fc16 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:23.665764 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafb88f-06fc-4275-b6f4-9f668e24513d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:23.666442 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafb88f-06fc-4275-b6f4-9f668e24513d None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:23.666756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafb88f-06fc-4275-b6f4-9f668e24513d None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:23.667041 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceafb88f-06fc-4275-b6f4-9f668e24513d None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:23.667772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceafb88f-06fc-4275-b6f4-9f668e24513d None admin] RBAC: Adding query filter params (name=network) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:23.668085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceafb88f-06fc-4275-b6f4-9f668e24513d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:23.670012 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceafb88f-06fc-4275-b6f4-9f668e24513d None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:23.678959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 395/777] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jul 20 09:00:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:23.684564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea5cc1-0d40-45cc-8526-82af6749bd58 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:23.714541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:23.856770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea5cc1-0d40-45cc-8526-82af6749bd58 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:23.891334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9305d-8b74-45ad-8b5e-af3a057087cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:23.891990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9305d-8b74-45ad-8b5e-af3a057087cd None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:23.892253 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9305d-8b74-45ad-8b5e-af3a057087cd None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:23.892535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9305d-8b74-45ad-8b5e-af3a057087cd None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:23.893242 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9305d-8b74-45ad-8b5e-af3a057087cd None admin] RBAC: Adding query filter params (type=network) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:23.893544 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9305d-8b74-45ad-8b5e-af3a057087cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:23.894939 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9305d-8b74-45ad-8b5e-af3a057087cd None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:23.900368 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 383/778] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jul 20 09:00:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:23.906704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafb88f-06fc-4275-b6f4-9f668e24513d None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:23.943077 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:24.044561 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceafb88f-06fc-4275-b6f4-9f668e24513d None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:24.090263 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27938d17-0b85-4d92-878a-ffb9663b7126 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:24.091024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27938d17-0b85-4d92-878a-ffb9663b7126 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:24.091565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27938d17-0b85-4d92-878a-ffb9663b7126 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:24.093424 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 384/779] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:24.096626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b59bc-bbb6-4f74-b13c-f18bd6dbd8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:24.097225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b59bc-bbb6-4f74-b13c-f18bd6dbd8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:24.097506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b59bc-bbb6-4f74-b13c-f18bd6dbd8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:24.097782 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b59bc-bbb6-4f74-b13c-f18bd6dbd8e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:24.098576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b59bc-bbb6-4f74-b13c-f18bd6dbd8e3 None admin] RBAC: Adding query filter params (interface=public, service_id=ebf704b5d5eb4c84b61fdab5a3cf0491, region_id=RegionOne) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:24.098917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b59bc-bbb6-4f74-b13c-f18bd6dbd8e3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:24.100281 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b59bc-bbb6-4f74-b13c-f18bd6dbd8e3 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:24.100549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ef928-4643-4cd5-8901-e53526be694b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:24.101062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ef928-4643-4cd5-8901-e53526be694b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:24.101564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ef928-4643-4cd5-8901-e53526be694b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:24.115159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 396/780] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Jul 20 09:00:23 2022] GET /identity/v3/endpoints?service_id=ebf704b5d5eb4c84b61fdab5a3cf0491&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:24.121326 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b59bc-bbb6-4f74-b13c-f18bd6dbd8e3 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:24.135153 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-714ef928-4643-4cd5-8901-e53526be694b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:24.138924 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-714ef928-4643-4cd5-8901-e53526be694b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:24.252616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 385/781] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:24.258091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1df8e4-7cf6-4de4-89fd-325421f6e7f0 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:24.258091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1df8e4-7cf6-4de4-89fd-325421f6e7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:24.258091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1df8e4-7cf6-4de4-89fd-325421f6e7f0 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:24.258091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 386/782] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:24.266927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e5910-d766-4e19-9f59-72df93d122ad None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:24.267214 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e5910-d766-4e19-9f59-72df93d122ad None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:24.268301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e5910-d766-4e19-9f59-72df93d122ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:24.287111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:24.293470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b59bc-bbb6-4f74-b13c-f18bd6dbd8e3 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:24.295360 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-fe9e5910-d766-4e19-9f59-72df93d122ad None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:24.297640 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-fe9e5910-d766-4e19-9f59-72df93d122ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:24.299844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:24.331835 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f740f1d-73ec-47f0-afc6-69ab62057f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:24.332952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f740f1d-73ec-47f0-afc6-69ab62057f73 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:24.333341 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f740f1d-73ec-47f0-afc6-69ab62057f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:24.333765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f740f1d-73ec-47f0-afc6-69ab62057f73 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:24.334536 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f740f1d-73ec-47f0-afc6-69ab62057f73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:24.336143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f740f1d-73ec-47f0-afc6-69ab62057f73 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:24.342042 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 397/783] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jul 20 09:00:24 2022] GET /identity/v3/services => generated 772 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:24.343731 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f740f1d-73ec-47f0-afc6-69ab62057f73 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:24.391816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 387/784] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:24.427634 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e5910-d766-4e19-9f59-72df93d122ad None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:24.532283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f740f1d-73ec-47f0-afc6-69ab62057f73 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:24.568936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba005c-9b71-4312-a9e9-95371ed152e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:24.570140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba005c-9b71-4312-a9e9-95371ed152e5 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:24.570429 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba005c-9b71-4312-a9e9-95371ed152e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:24.570831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba005c-9b71-4312-a9e9-95371ed152e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:24.580048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e5910-d766-4e19-9f59-72df93d122ad None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:24.582163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba005c-9b71-4312-a9e9-95371ed152e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:24.589979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-83ba005c-9b71-4312-a9e9-95371ed152e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:00:24.589979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:24.589979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:24.589979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:24.589979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:24.589979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:24.589979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:24.589979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:24.589979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:24.589979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:24.589979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:24.589979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:24.589979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:24.589979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:24.589979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:24.589979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:24.591529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:24.591529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:24.591529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:24.591529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:24.591529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:24.591529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:24.591529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:24.591529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:24.591529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:24.591529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:24.591529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:24.591529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:24.591529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:24.591529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:24.591529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:24.591529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:24.592373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:24.592373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:24.592373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:24.592373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:24.592373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:24.592373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:24.592373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:24.592373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:24.592373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:24.592373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:24.592373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:00:24.592373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:24.593005 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 398/785] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:00:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:24.598239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba005c-9b71-4312-a9e9-95371ed152e5 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:24.618529 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b3f5b-e579-4187-81b6-072e80b4cdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:24.619411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b3f5b-e579-4187-81b6-072e80b4cdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:24.619695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b3f5b-e579-4187-81b6-072e80b4cdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:24.619973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b3f5b-e579-4187-81b6-072e80b4cdc5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:24.628112 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:24.632227 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b3f5b-e579-4187-81b6-072e80b4cdc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:24.641777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-fe4b3f5b-e579-4187-81b6-072e80b4cdc5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:24.641777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:24.641777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:24.641777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:24.641777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:24.641777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:24.641777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:24.641777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:24.641777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:24.641777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:24.641777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:24.641777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:24.641777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:24.641777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:24.641777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:24.641777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:24.642816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:24.642816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:24.642816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:24.642816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:24.642816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:24.642816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:24.642816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:24.642816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:24.642816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:24.642816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:24.642816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:24.642816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:24.642816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:24.642816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:24.642816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:24.642816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:24.643681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:24.643681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:24.643681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:24.643681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:24.643681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:24.643681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:24.643681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:24.643681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:24.643681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:24.643681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:24.643681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:24.643681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:24.643681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 388/786] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:00:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:24.651228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:24.652660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b3f5b-e579-4187-81b6-072e80b4cdc5 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:24.722419 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba005c-9b71-4312-a9e9-95371ed152e5 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:24.760409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951eb2e0-6d96-4db9-9191-b80bec49807a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:24.761177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951eb2e0-6d96-4db9-9191-b80bec49807a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:24.761487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951eb2e0-6d96-4db9-9191-b80bec49807a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:24.761832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951eb2e0-6d96-4db9-9191-b80bec49807a None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:24.762583 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951eb2e0-6d96-4db9-9191-b80bec49807a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:24.763497 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951eb2e0-6d96-4db9-9191-b80bec49807a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:24.765014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951eb2e0-6d96-4db9-9191-b80bec49807a None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:24.768669 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b3f5b-e579-4187-81b6-072e80b4cdc5 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:24.774329 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 399/787] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:00:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:24.779833 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951eb2e0-6d96-4db9-9191-b80bec49807a None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:24.805706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85bb2e3-7337-4797-9d33-841a6a3bc355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:24.806486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bb2e3-7337-4797-9d33-841a6a3bc355 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:24.806817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bb2e3-7337-4797-9d33-841a6a3bc355 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:24.807103 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85bb2e3-7337-4797-9d33-841a6a3bc355 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:24.807863 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85bb2e3-7337-4797-9d33-841a6a3bc355 None admin] RBAC: Adding query filter params (name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:24.808178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85bb2e3-7337-4797-9d33-841a6a3bc355 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:24.810346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85bb2e3-7337-4797-9d33-841a6a3bc355 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:24.819706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 389/788] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 09:00:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:24.830697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85bb2e3-7337-4797-9d33-841a6a3bc355 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:24.914054 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:24.937536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:24.948730 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951eb2e0-6d96-4db9-9191-b80bec49807a None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:25.005175 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85bb2e3-7337-4797-9d33-841a6a3bc355 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:25.016473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf00fc-8853-4199-b164-0570650c88ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:25.017439 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf00fc-8853-4199-b164-0570650c88ef None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:25.017723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf00fc-8853-4199-b164-0570650c88ef None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:25.017998 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf00fc-8853-4199-b164-0570650c88ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:25.035244 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf00fc-8853-4199-b164-0570650c88ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:25.036706 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf00fc-8853-4199-b164-0570650c88ef None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:25.072719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-9faf00fc-8853-4199-b164-0570650c88ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:25.072719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:25.072719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:25.072719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:25.072719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:25.072719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:25.072719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:25.072719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:25.072719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:25.072719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:25.072719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:25.072719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:25.072719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:25.072719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:25.072719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:25.075146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:25.077590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:25.077590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:25.077590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:25.077590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:25.077590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 400/789] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:25.079796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cc3d2-d003-4d05-8082-babad0e97a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:25.080898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cc3d2-d003-4d05-8082-babad0e97a12 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:25.081313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cc3d2-d003-4d05-8082-babad0e97a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:25.081718 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cc3d2-d003-4d05-8082-babad0e97a12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:25.084515 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf00fc-8853-4199-b164-0570650c88ef None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:25.105392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cc3d2-d003-4d05-8082-babad0e97a12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:25.107740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cc3d2-d003-4d05-8082-babad0e97a12 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:25.147547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-d95cc3d2-d003-4d05-8082-babad0e97a12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:25.147547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:25.147547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:25.147547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:25.147547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:25.147547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:25.147547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:25.147547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:25.147547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:25.147547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:25.147547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:25.147547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:25.147547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:25.147547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:25.147547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:25.148653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:25.149619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:25.149619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:25.149619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:25.149619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:25.151490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 390/790] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:25.157264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cc3d2-d003-4d05-8082-babad0e97a12 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:25.215601 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf00fc-8853-4199-b164-0570650c88ef None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:25.257289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacc681-ae33-4b97-a43a-ed726acb2904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:25.258340 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacc681-ae33-4b97-a43a-ed726acb2904 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:25.258878 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacc681-ae33-4b97-a43a-ed726acb2904 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:25.259380 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacc681-ae33-4b97-a43a-ed726acb2904 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:25.260305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacc681-ae33-4b97-a43a-ed726acb2904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:25.260892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacc681-ae33-4b97-a43a-ed726acb2904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:25.262525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacc681-ae33-4b97-a43a-ed726acb2904 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:25.270971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:25.291504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 401/791] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:25.298351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacc681-ae33-4b97-a43a-ed726acb2904 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:25.337142 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cc3d2-d003-4d05-8082-babad0e97a12 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:25.369066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:25.408714 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f4832-effa-413e-8442-72fa31f06c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:25.409640 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f4832-effa-413e-8442-72fa31f06c8a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:25.410106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f4832-effa-413e-8442-72fa31f06c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:25.410641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f4832-effa-413e-8442-72fa31f06c8a None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:25.411638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f4832-effa-413e-8442-72fa31f06c8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:25.412097 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f4832-effa-413e-8442-72fa31f06c8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:25.413615 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f4832-effa-413e-8442-72fa31f06c8a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:25.438943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 391/792] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:25.447691 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f4832-effa-413e-8442-72fa31f06c8a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:25.469279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacc681-ae33-4b97-a43a-ed726acb2904 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:25.532959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a16d4-383a-4353-a05d-f2262342f2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:25.534494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a16d4-383a-4353-a05d-f2262342f2ec None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:25.535018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a16d4-383a-4353-a05d-f2262342f2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:25.535306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a16d4-383a-4353-a05d-f2262342f2ec None admin] PATH_INFO: `/v3/users/swift` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:25.556213 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a16d4-383a-4353-a05d-f2262342f2ec None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:25.557710 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a16d4-383a-4353-a05d-f2262342f2ec None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:25.591554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-cb1a16d4-383a-4353-a05d-f2262342f2ec None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:00:25.591554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:25.591554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:25.591554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:25.591554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:25.591554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:25.591554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:25.591554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:25.591554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:25.591554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:25.591554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:25.591554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:25.591554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:25.591554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:25.591554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:25.591554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:25.592557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:25.593513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:25.593513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:25.593513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:25.593513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:25.593513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:25.593513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:25.593513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:25.593513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:25.593513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:00:25.593513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:25.596251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 402/793] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:00:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:25.598719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f4832-effa-413e-8442-72fa31f06c8a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:25.601784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a16d4-383a-4353-a05d-f2262342f2ec None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:25.643981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7483b53-c115-4ecf-aff4-0a969c139d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:25.644792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7483b53-c115-4ecf-aff4-0a969c139d48 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:25.645076 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7483b53-c115-4ecf-aff4-0a969c139d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:25.645351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7483b53-c115-4ecf-aff4-0a969c139d48 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:25.657103 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7483b53-c115-4ecf-aff4-0a969c139d48 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:25.658885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7483b53-c115-4ecf-aff4-0a969c139d48 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:25.685982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-b7483b53-c115-4ecf-aff4-0a969c139d48 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:00:25.685982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:25.685982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:25.685982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:25.685982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:25.685982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:25.685982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:25.685982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:25.685982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:25.685982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:25.685982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:25.685982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:25.685982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:25.685982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:25.685982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:25.685982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:25.687011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:25.687941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:25.687941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:25.687941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:25.687941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:25.687941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:25.687941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:25.687941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:25.687941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:25.687941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:00:25.687941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:25.691751 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 392/794] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jul 20 09:00:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:25.692576 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718db4ed-bb9e-4a77-8f55-3b074cdc9319 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:25.692701 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718db4ed-bb9e-4a77-8f55-3b074cdc9319 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:25.692954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718db4ed-bb9e-4a77-8f55-3b074cdc9319 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:25.694620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 393/795] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:25.698529 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718db4ed-bb9e-4a77-8f55-3b074cdc9319 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:25.704201 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:25.736029 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:25.758065 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a16d4-383a-4353-a05d-f2262342f2ec None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:25.794922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374c3f2-f8c6-4df9-ac73-b8f3e05915a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:25.795609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374c3f2-f8c6-4df9-ac73-b8f3e05915a1 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:25.795899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374c3f2-f8c6-4df9-ac73-b8f3e05915a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:25.796178 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374c3f2-f8c6-4df9-ac73-b8f3e05915a1 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:25.797019 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9374c3f2-f8c6-4df9-ac73-b8f3e05915a1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:25.797332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9374c3f2-f8c6-4df9-ac73-b8f3e05915a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:25.798799 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9374c3f2-f8c6-4df9-ac73-b8f3e05915a1 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:25.816881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 403/796] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Jul 20 09:00:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:25.818121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c22c30-e996-496a-8d52-3c4d4c62881b None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:25.818414 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c22c30-e996-496a-8d52-3c4d4c62881b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:25.818724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c22c30-e996-496a-8d52-3c4d4c62881b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:25.823179 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718db4ed-bb9e-4a77-8f55-3b074cdc9319 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:25.836801 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-e7c22c30-e996-496a-8d52-3c4d4c62881b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:25.837989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e7c22c30-e996-496a-8d52-3c4d4c62881b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:25.860262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cea64-3077-45bb-8b55-dbccb44d3232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:25.860945 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cea64-3077-45bb-8b55-dbccb44d3232 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:25.861232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cea64-3077-45bb-8b55-dbccb44d3232 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:25.861506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cea64-3077-45bb-8b55-dbccb44d3232 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:25.862351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cea64-3077-45bb-8b55-dbccb44d3232 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:25.862666 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cea64-3077-45bb-8b55-dbccb44d3232 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:25.864133 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cea64-3077-45bb-8b55-dbccb44d3232 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:25.886411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 394/797] 10.222.0.44 () {56 vars in 1149 bytes} [Wed Jul 20 09:00:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:25.886411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859b3ed-f1f1-4278-8b06-6cc38c113b48 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:25.886411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859b3ed-f1f1-4278-8b06-6cc38c113b48 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:25.886411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859b3ed-f1f1-4278-8b06-6cc38c113b48 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:25.886411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 395/798] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:25.886411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859b3ed-f1f1-4278-8b06-6cc38c113b48 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:25.917592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 404/799] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:25.918708 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c22c30-e996-496a-8d52-3c4d4c62881b None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:25.977386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:25.995600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:26.008681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859b3ed-f1f1-4278-8b06-6cc38c113b48 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:26.039523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c22c30-e996-496a-8d52-3c4d4c62881b None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:26.045532 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec60ed-41e3-415e-ac46-0a333db42336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:26.046324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec60ed-41e3-415e-ac46-0a333db42336 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:26.046604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec60ed-41e3-415e-ac46-0a333db42336 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:26.046905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec60ed-41e3-415e-ac46-0a333db42336 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:26.057857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec60ed-41e3-415e-ac46-0a333db42336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:26.059560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec60ed-41e3-415e-ac46-0a333db42336 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:26.074231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2363b1-e352-457e-86e7-eca147f74836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:26.075051 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2363b1-e352-457e-86e7-eca147f74836 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:26.075988 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2363b1-e352-457e-86e7-eca147f74836 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:26.076337 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2363b1-e352-457e-86e7-eca147f74836 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:26.079019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-84ec60ed-41e3-415e-ac46-0a333db42336 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:26.079019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:26.079019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:26.079019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:26.079019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:26.079019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:26.079019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:26.079019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:26.079019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:26.079019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:26.079019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:26.079019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:26.079019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:26.079019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:26.079019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:26.079978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:26.080870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:26.080870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:26.080870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:26.080870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:26.080870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 396/800] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:26.082138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235c277-344a-4d48-ac0d-2b2def41c8ef None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:26.082138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235c277-344a-4d48-ac0d-2b2def41c8ef None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:26.082138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235c277-344a-4d48-ac0d-2b2def41c8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:26.092437 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2363b1-e352-457e-86e7-eca147f74836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:26.093945 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2363b1-e352-457e-86e7-eca147f74836 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:26.098561 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-c235c277-344a-4d48-ac0d-2b2def41c8ef None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:26.099764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c235c277-344a-4d48-ac0d-2b2def41c8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:26.112014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-5d2363b1-e352-457e-86e7-eca147f74836 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:26.112014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:26.112014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:26.112014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:26.112014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:26.112014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:26.112014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:26.112014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:26.112014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:26.112014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:26.112014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:26.112014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:26.112014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:26.112014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:26.112014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:26.112918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:26.113794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:26.113794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:26.113794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:26.113794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:26.113794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 405/801] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:26.114063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2363b1-e352-457e-86e7-eca147f74836 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:26.179592 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 397/802] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:26.180517 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235c277-344a-4d48-ac0d-2b2def41c8ef None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:26.227484 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:26.235999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2363b1-e352-457e-86e7-eca147f74836 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:26.261650 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:26.272402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce9521-1c24-4abb-8a58-df12566144fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:26.273199 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce9521-1c24-4abb-8a58-df12566144fd None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:26.273482 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce9521-1c24-4abb-8a58-df12566144fd None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:26.273756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce9521-1c24-4abb-8a58-df12566144fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:26.286150 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce9521-1c24-4abb-8a58-df12566144fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:26.293871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-28ce9521-1c24-4abb-8a58-df12566144fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:00:26.293871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:26.293871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:26.293871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:26.293871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:26.293871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:26.293871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:26.293871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:26.293871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:26.293871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:26.293871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:26.293871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:26.293871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:26.293871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:26.293871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:26.293871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:26.295013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:26.295013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:26.295013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:26.295013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:26.295013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:26.295013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:26.295013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:26.295013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:26.295013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:26.295013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:26.295013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:26.295013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:26.295013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:26.295013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:26.295013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:26.295013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:26.295853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:26.295853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:26.295853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:26.295853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:26.295853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:26.295853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:26.295853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:26.295853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:26.295853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:26.295853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:26.295853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:00:26.295853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:26.296487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235c277-344a-4d48-ac0d-2b2def41c8ef None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:26.296561 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 406/803] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:00:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:26.296561 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce9521-1c24-4abb-8a58-df12566144fd None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:26.335281 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167535d0-bc06-4f9f-a073-4034beb09045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:26.335938 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167535d0-bc06-4f9f-a073-4034beb09045 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:26.336220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167535d0-bc06-4f9f-a073-4034beb09045 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:26.336499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167535d0-bc06-4f9f-a073-4034beb09045 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:26.337221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167535d0-bc06-4f9f-a073-4034beb09045 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:26.337533 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167535d0-bc06-4f9f-a073-4034beb09045 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:26.338921 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167535d0-bc06-4f9f-a073-4034beb09045 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:26.351240 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 398/804] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:26.351978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167535d0-bc06-4f9f-a073-4034beb09045 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:26.419693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce9521-1c24-4abb-8a58-df12566144fd None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:26.457632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167535d0-bc06-4f9f-a073-4034beb09045 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:26.461689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a30e2-8873-463d-9a08-41d55469ec62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:26.462339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a30e2-8873-463d-9a08-41d55469ec62 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:26.462621 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a30e2-8873-463d-9a08-41d55469ec62 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:26.462926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a30e2-8873-463d-9a08-41d55469ec62 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:26.463647 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a30e2-8873-463d-9a08-41d55469ec62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:26.463951 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a30e2-8873-463d-9a08-41d55469ec62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:26.465320 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a30e2-8873-463d-9a08-41d55469ec62 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:26.485192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 407/805] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:26.485192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a30e2-8873-463d-9a08-41d55469ec62 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:26.495348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:26.500077 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:26.501510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921f317-79e4-435e-a0cc-79fefe117d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:26.502300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921f317-79e4-435e-a0cc-79fefe117d0e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:26.502580 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921f317-79e4-435e-a0cc-79fefe117d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:26.502887 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921f317-79e4-435e-a0cc-79fefe117d0e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:26.503709 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921f317-79e4-435e-a0cc-79fefe117d0e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:26.504910 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921f317-79e4-435e-a0cc-79fefe117d0e None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:26.518815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-9921f317-79e4-435e-a0cc-79fefe117d0e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:00:26.518815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:26.518815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:26.518815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:26.518815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:26.518815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:26.518815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:26.518815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:26.518815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:00:26.518815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:26.518815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:26.518815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:26.518815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:26.518815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:26.518815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:26.518815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:26.519747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:26.519747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:26.519747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:26.519747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:26.519747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:26.519747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:26.519747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:26.519747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:26.519747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:26.519747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:26.519747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:26.519747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:26.519747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:26.519747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:26.519747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:26.519747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:26.525408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:26.525408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:26.525408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:26.525408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:26.525408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:26.525408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:26.525408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:26.525408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:26.525408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:26.525408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:26.525408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:26.525408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:26.525408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:26.525408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 09:00:26.525408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:26.525408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 399/806] 10.222.0.44 () {56 vars in 1129 bytes} [Wed Jul 20 09:00:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:26.526272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921f317-79e4-435e-a0cc-79fefe117d0e None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:26.601140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a30e2-8873-463d-9a08-41d55469ec62 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:26.639950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c029f0-d6dc-49a1-91d6-ccc143e7ed5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:26.640280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921f317-79e4-435e-a0cc-79fefe117d0e None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:26.640624 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c029f0-d6dc-49a1-91d6-ccc143e7ed5a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:26.640910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c029f0-d6dc-49a1-91d6-ccc143e7ed5a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:26.641187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c029f0-d6dc-49a1-91d6-ccc143e7ed5a None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:26.642216 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c029f0-d6dc-49a1-91d6-ccc143e7ed5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:26.643006 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c029f0-d6dc-49a1-91d6-ccc143e7ed5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:26.644515 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c029f0-d6dc-49a1-91d6-ccc143e7ed5a None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:26.658030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 408/807] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:00:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:26.658030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c029f0-d6dc-49a1-91d6-ccc143e7ed5a None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:26.681650 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ac3a3-38d2-43f7-97fd-b3be59c849a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:26.682482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ac3a3-38d2-43f7-97fd-b3be59c849a4 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:26.682841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ac3a3-38d2-43f7-97fd-b3be59c849a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:26.683126 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ac3a3-38d2-43f7-97fd-b3be59c849a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:26.691381 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ac3a3-38d2-43f7-97fd-b3be59c849a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:26.692899 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ac3a3-38d2-43f7-97fd-b3be59c849a4 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:26.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-7d6ac3a3-38d2-43f7-97fd-b3be59c849a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:26.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:26.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:26.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:26.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:26.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:26.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:26.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:26.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:26.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:26.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:26.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:26.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:26.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:26.709809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:26.710772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:26.711784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:26.711784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:26.711784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:26.711784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:26.711784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:26.711784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:26.711784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:26.711784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:26.711784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 400/808] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:26.712277 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ac3a3-38d2-43f7-97fd-b3be59c849a4 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:26.746999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:26.772455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:26.775010 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c029f0-d6dc-49a1-91d6-ccc143e7ed5a None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:26.812704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15023b-73dc-4069-befe-7a598c4ea866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:26.813502 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15023b-73dc-4069-befe-7a598c4ea866 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:26.813788 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15023b-73dc-4069-befe-7a598c4ea866 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:26.814065 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15023b-73dc-4069-befe-7a598c4ea866 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:26.820823 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ac3a3-38d2-43f7-97fd-b3be59c849a4 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:26.821390 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15023b-73dc-4069-befe-7a598c4ea866 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:26.822872 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15023b-73dc-4069-befe-7a598c4ea866 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:26.837434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-ed15023b-73dc-4069-befe-7a598c4ea866 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:26.837434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:26.837434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:26.837434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:26.837434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:26.837434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:26.837434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:26.837434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:26.837434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:26.837434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:26.837434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:26.837434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:26.837434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:26.837434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:26.837434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:26.838365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:26.839317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:26.839317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:26.839317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:26.839317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:26.839317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:26.839317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:26.839317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:26.839317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:26.839317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 409/809] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:26.839809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15023b-73dc-4069-befe-7a598c4ea866 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:26.863872 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf975d9-0c60-42a6-8a92-bfaff879c422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:26.864547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf975d9-0c60-42a6-8a92-bfaff879c422 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:26.864835 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf975d9-0c60-42a6-8a92-bfaff879c422 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:26.865122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf975d9-0c60-42a6-8a92-bfaff879c422 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:26.865838 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf975d9-0c60-42a6-8a92-bfaff879c422 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:26.866151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf975d9-0c60-42a6-8a92-bfaff879c422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:26.867550 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf975d9-0c60-42a6-8a92-bfaff879c422 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:26.871911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 401/810] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Jul 20 09:00:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:26.872863 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf975d9-0c60-42a6-8a92-bfaff879c422 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:26.940997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15023b-73dc-4069-befe-7a598c4ea866 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:26.971800 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf975d9-0c60-42a6-8a92-bfaff879c422 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:26.979769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3320f-4173-4b64-bcba-474b6293d7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:26.980562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3320f-4173-4b64-bcba-474b6293d7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:26.980844 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3320f-4173-4b64-bcba-474b6293d7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:26.981121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3320f-4173-4b64-bcba-474b6293d7b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:26.984097 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:26.991572 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:26.994004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3320f-4173-4b64-bcba-474b6293d7b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:26.995475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3320f-4173-4b64-bcba-474b6293d7b4 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:27.007007 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e7aae-2d33-425a-9903-2bdbe63b68e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:27.007660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e7aae-2d33-425a-9903-2bdbe63b68e5 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:27.007940 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e7aae-2d33-425a-9903-2bdbe63b68e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:27.008215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e7aae-2d33-425a-9903-2bdbe63b68e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:27.008990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e7aae-2d33-425a-9903-2bdbe63b68e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:27.009298 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e7aae-2d33-425a-9903-2bdbe63b68e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:27.010716 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4e7aae-2d33-425a-9903-2bdbe63b68e5 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:27.013741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-67f3320f-4173-4b64-bcba-474b6293d7b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:27.013741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:27.013741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:27.013741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:27.013741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:27.013741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:27.013741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:27.013741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:27.013741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:27.013741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:27.013741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:27.013741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:27.013741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:27.013741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:27.013741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:27.014821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:27.015758 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:27.015758 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:27.015758 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:27.015758 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:27.015758 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 410/811] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:27.016481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847d1b6-8f7d-489e-9de8-258b9bc44ca1 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:27.016768 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847d1b6-8f7d-489e-9de8-258b9bc44ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:27.017046 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847d1b6-8f7d-489e-9de8-258b9bc44ca1 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:27.018301 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 411/812] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:27.019016 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847d1b6-8f7d-489e-9de8-258b9bc44ca1 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:27.030453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 402/813] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:27.030453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e7aae-2d33-425a-9903-2bdbe63b68e5 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:27.121946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847d1b6-8f7d-489e-9de8-258b9bc44ca1 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:27.124734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e7aae-2d33-425a-9903-2bdbe63b68e5 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:27.156008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55606d5-4697-4e2f-aee8-f29011eab88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:27.156766 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55606d5-4697-4e2f-aee8-f29011eab88a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:27.157116 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55606d5-4697-4e2f-aee8-f29011eab88a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:27.157461 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55606d5-4697-4e2f-aee8-f29011eab88a None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:27.158063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5c486-b998-47c9-9956-ffc3cbbaf930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:27.158313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55606d5-4697-4e2f-aee8-f29011eab88a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:27.158629 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55606d5-4697-4e2f-aee8-f29011eab88a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:27.158769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5c486-b998-47c9-9956-ffc3cbbaf930 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:27.159012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5c486-b998-47c9-9956-ffc3cbbaf930 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:27.159286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5c486-b998-47c9-9956-ffc3cbbaf930 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:27.159986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5c486-b998-47c9-9956-ffc3cbbaf930 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:27.160079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55606d5-4697-4e2f-aee8-f29011eab88a None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:27.160289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5c486-b998-47c9-9956-ffc3cbbaf930 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:27.161617 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5c486-b998-47c9-9956-ffc3cbbaf930 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:27.166580 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 403/814] 10.222.0.44 () {56 vars in 1124 bytes} [Wed Jul 20 09:00:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:27.167175 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5c486-b998-47c9-9956-ffc3cbbaf930 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:27.172223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 412/815] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:27.173616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3df017-aacf-43a6-bcdf-b502e3d73f33 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:27.173906 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3df017-aacf-43a6-bcdf-b502e3d73f33 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:27.174185 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3df017-aacf-43a6-bcdf-b502e3d73f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:27.190637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-0f3df017-aacf-43a6-bcdf-b502e3d73f33 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:27.191847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0f3df017-aacf-43a6-bcdf-b502e3d73f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:27.218844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:27.236663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:27.260745 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 413/816] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:27.261553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3df017-aacf-43a6-bcdf-b502e3d73f33 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:27.282745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5c486-b998-47c9-9956-ffc3cbbaf930 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:27.316965 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73c8f5-9b4a-4842-9271-382e45677687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:27.317632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73c8f5-9b4a-4842-9271-382e45677687 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:27.317914 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73c8f5-9b4a-4842-9271-382e45677687 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:27.318191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73c8f5-9b4a-4842-9271-382e45677687 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:27.318948 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73c8f5-9b4a-4842-9271-382e45677687 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:27.319519 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73c8f5-9b4a-4842-9271-382e45677687 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:27.320894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73c8f5-9b4a-4842-9271-382e45677687 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:27.334303 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 404/817] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:27.335129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73c8f5-9b4a-4842-9271-382e45677687 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:27.367678 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3df017-aacf-43a6-bcdf-b502e3d73f33 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:27.401992 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545daaf-4d98-498a-b95f-e5c24727fa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:27.402911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545daaf-4d98-498a-b95f-e5c24727fa03 None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:27.403196 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545daaf-4d98-498a-b95f-e5c24727fa03 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:27.404184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545daaf-4d98-498a-b95f-e5c24727fa03 None admin] PATH_INFO: `/v3/projects/a498284274454d25b1b118a88996e522/users/03b4375e6ead42fb8d77a7b30617cb95/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:27.406291 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545daaf-4d98-498a-b95f-e5c24727fa03 None admin] RBAC: Authorizing `identity:create_grant(project_id=a498284274454d25b1b118a88996e522, user_id=03b4375e6ead42fb8d77a7b30617cb95, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:27.407760 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545daaf-4d98-498a-b95f-e5c24727fa03 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:27.421071 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 414/818] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 09:00:27 2022] PUT /identity/v3/projects/a498284274454d25b1b118a88996e522/users/03b4375e6ead42fb8d77a7b30617cb95/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:00:27.421829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545daaf-4d98-498a-b95f-e5c24727fa03 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:27.437408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73c8f5-9b4a-4842-9271-382e45677687 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:27.441218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:27.468200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:27.481916 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c7538-cc35-4c2c-983e-4192b9423534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:27.482465 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c7538-cc35-4c2c-983e-4192b9423534 None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:27.482842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c7538-cc35-4c2c-983e-4192b9423534 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:27.483299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c7538-cc35-4c2c-983e-4192b9423534 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:27.484125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c7538-cc35-4c2c-983e-4192b9423534 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:27.485777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c7538-cc35-4c2c-983e-4192b9423534 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:27.513862 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 405/819] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jul 20 09:00:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:00:27.515431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c7538-cc35-4c2c-983e-4192b9423534 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:27.543683 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545daaf-4d98-498a-b95f-e5c24727fa03 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:27.598979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa16100-6859-42b3-a5f3-01473e6cf142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:27.599950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa16100-6859-42b3-a5f3-01473e6cf142 None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:27.600233 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa16100-6859-42b3-a5f3-01473e6cf142 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:27.600508 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa16100-6859-42b3-a5f3-01473e6cf142 None admin] PATH_INFO: `/v3/projects/a498284274454d25b1b118a88996e522/users/6b8db4a7ee284b1aa96d4517dd381e71/roles/ae949d8bea4b40d9b1a9fa207d539663` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:27.619414 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa16100-6859-42b3-a5f3-01473e6cf142 None admin] RBAC: Authorizing `identity:create_grant(project_id=a498284274454d25b1b118a88996e522, user_id=6b8db4a7ee284b1aa96d4517dd381e71, role_id=ae949d8bea4b40d9b1a9fa207d539663)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:27.621033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa16100-6859-42b3-a5f3-01473e6cf142 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:27.639241 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 415/820] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 09:00:27 2022] PUT /identity/v3/projects/a498284274454d25b1b118a88996e522/users/6b8db4a7ee284b1aa96d4517dd381e71/roles/ae949d8bea4b40d9b1a9fa207d539663 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:00:27.639798 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa16100-6859-42b3-a5f3-01473e6cf142 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:27.673787 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c7538-cc35-4c2c-983e-4192b9423534 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:27.715601 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d844a6-e4fb-4d08-9636-07b0bb2ec4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:27.716625 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d844a6-e4fb-4d08-9636-07b0bb2ec4ef None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:27.717179 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d844a6-e4fb-4d08-9636-07b0bb2ec4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:27.717635 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d844a6-e4fb-4d08-9636-07b0bb2ec4ef None admin] PATH_INFO: `/v3/services/network` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:27.718411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d844a6-e4fb-4d08-9636-07b0bb2ec4ef None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:27.719974 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d844a6-e4fb-4d08-9636-07b0bb2ec4ef None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-b0d844a6-e4fb-4d08-9636-07b0bb2ec4ef None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:27.733106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:27.734247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:27.734247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:27.734247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:27.734247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:27.734247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:27.734247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:27.734247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:27.734247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:27.734247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:27.734247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:27.734247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:27.734247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:27.734247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:27.734247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:27.734247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:27.734247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:27.735238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:27.735238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:27.735238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:27.735238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:27.735238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:27.735238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:27.735238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:27.735238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:27.735238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:27.735238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:27.735238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:27.735238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:27.735238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 09:00:27.735238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:27.735238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 406/821] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:27 2022] GET /identity/v3/services/network => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:27.743789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d844a6-e4fb-4d08-9636-07b0bb2ec4ef None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:27.777176 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:27.788511 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:27.803719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa16100-6859-42b3-a5f3-01473e6cf142 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:27.853364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c40227-59dc-4ea5-8718-26bd18d4cebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:27.854178 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c40227-59dc-4ea5-8718-26bd18d4cebc None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:27.854457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c40227-59dc-4ea5-8718-26bd18d4cebc None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:27.854764 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c40227-59dc-4ea5-8718-26bd18d4cebc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:27.869606 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c40227-59dc-4ea5-8718-26bd18d4cebc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:27.873456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c40227-59dc-4ea5-8718-26bd18d4cebc None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:27.894415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-e4c40227-59dc-4ea5-8718-26bd18d4cebc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:00:27.894415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:27.894415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:27.894415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:27.894415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:27.894415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:27.894415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:27.894415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:27.894415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:27.894415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:27.894415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:27.894415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:27.894415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:27.894415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:27.894415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:27.894415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:27.895496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:27.896435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:27.896435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:27.896435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:27.896435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:27.896435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:27.896435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:27.896435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:27.896435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:27.896435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:00:27.896435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:27.899564 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 416/822] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jul 20 09:00:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:27.905255 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c40227-59dc-4ea5-8718-26bd18d4cebc None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:27.930381 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d844a6-e4fb-4d08-9636-07b0bb2ec4ef None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:27.968594 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af55f60-91ce-4ed6-a4e3-671b0403fc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:27.969338 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af55f60-91ce-4ed6-a4e3-671b0403fc49 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:27.969624 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af55f60-91ce-4ed6-a4e3-671b0403fc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:27.969898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af55f60-91ce-4ed6-a4e3-671b0403fc49 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:27.970643 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af55f60-91ce-4ed6-a4e3-671b0403fc49 None admin] RBAC: Adding query filter params (name=network) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:27.971114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af55f60-91ce-4ed6-a4e3-671b0403fc49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:27.972665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af55f60-91ce-4ed6-a4e3-671b0403fc49 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:27.982832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 407/823] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jul 20 09:00:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:27.984359 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af55f60-91ce-4ed6-a4e3-671b0403fc49 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:28.017572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c40227-59dc-4ea5-8718-26bd18d4cebc None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:28.055491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e38f6-ad6b-4d40-8afc-1fbfb6bebb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:28.056138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e38f6-ad6b-4d40-8afc-1fbfb6bebb96 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:28.056521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e38f6-ad6b-4d40-8afc-1fbfb6bebb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:28.056801 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e38f6-ad6b-4d40-8afc-1fbfb6bebb96 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:28.057642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e38f6-ad6b-4d40-8afc-1fbfb6bebb96 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:28.057949 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e38f6-ad6b-4d40-8afc-1fbfb6bebb96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:28.060158 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e38f6-ad6b-4d40-8afc-1fbfb6bebb96 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:28.087056 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:28.094492 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af55f60-91ce-4ed6-a4e3-671b0403fc49 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:28.102784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 417/824] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jul 20 09:00:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:28.104209 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e38f6-ad6b-4d40-8afc-1fbfb6bebb96 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:28.107441 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:28.132316 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d356348-334c-4be8-8319-e1aa15763271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:28.132988 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d356348-334c-4be8-8319-e1aa15763271 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:28.133273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d356348-334c-4be8-8319-e1aa15763271 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:28.133551 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d356348-334c-4be8-8319-e1aa15763271 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:28.134360 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d356348-334c-4be8-8319-e1aa15763271 None admin] RBAC: Adding query filter params (type=network) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:28.134692 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d356348-334c-4be8-8319-e1aa15763271 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:28.136179 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d356348-334c-4be8-8319-e1aa15763271 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:28.146432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 408/825] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jul 20 09:00:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:28.147863 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d356348-334c-4be8-8319-e1aa15763271 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:28.266476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e38f6-ad6b-4d40-8afc-1fbfb6bebb96 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:28.273700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d356348-334c-4be8-8319-e1aa15763271 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:28.311069 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08177b3-9239-42ac-8414-af5df4badf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:28.311647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08177b3-9239-42ac-8414-af5df4badf36 None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:28.311931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08177b3-9239-42ac-8414-af5df4badf36 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:28.312209 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08177b3-9239-42ac-8414-af5df4badf36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:28.312954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08177b3-9239-42ac-8414-af5df4badf36 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:28.315996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08177b3-9239-42ac-8414-af5df4badf36 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:28.338963 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 409/826] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jul 20 09:00:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:00:28.368031 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa31c3-23ab-410a-ada9-9cc734481668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:28.368799 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa31c3-23ab-410a-ada9-9cc734481668 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:28.369079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa31c3-23ab-410a-ada9-9cc734481668 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:28.369358 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa31c3-23ab-410a-ada9-9cc734481668 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:28.454516 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:28.462733 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaa31c3-23ab-410a-ada9-9cc734481668 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:28.464571 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaa31c3-23ab-410a-ada9-9cc734481668 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:28.577086 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 418/827] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:00:28 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:00:28.582528 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08177b3-9239-42ac-8414-af5df4badf36 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:28.594936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:28.693153 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08177b3-9239-42ac-8414-af5df4badf36 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:28.728493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d48268-98da-4067-bd79-66755c7d8b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:28.729288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d48268-98da-4067-bd79-66755c7d8b47 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:28.729564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d48268-98da-4067-bd79-66755c7d8b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:28.729832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d48268-98da-4067-bd79-66755c7d8b47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:28.741455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d48268-98da-4067-bd79-66755c7d8b47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:28.742942 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d48268-98da-4067-bd79-66755c7d8b47 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:28.760782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-c7d48268-98da-4067-bd79-66755c7d8b47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:28.760782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:28.760782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:28.760782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:28.760782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:28.760782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:28.760782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:28.760782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:28.760782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:28.760782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:28.760782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:28.760782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:28.760782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:28.760782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:28.760782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:28.761756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:28.763268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:28.763268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:28.763268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:28.763268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:28.763268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 410/828] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:28.769944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa31c3-23ab-410a-ada9-9cc734481668 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:28.880815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:28.884347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa31c3-23ab-410a-ada9-9cc734481668 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:28.920795 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823f5c5-70cd-443a-af87-56d937965e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:28.921461 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823f5c5-70cd-443a-af87-56d937965e36 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:28.921747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823f5c5-70cd-443a-af87-56d937965e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:28.922028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823f5c5-70cd-443a-af87-56d937965e36 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:28.922779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823f5c5-70cd-443a-af87-56d937965e36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:28.923092 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823f5c5-70cd-443a-af87-56d937965e36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:28.924461 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823f5c5-70cd-443a-af87-56d937965e36 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:28.936282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 419/829] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:28.941601 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d48268-98da-4067-bd79-66755c7d8b47 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:29.004170 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:29.053824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d48268-98da-4067-bd79-66755c7d8b47 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:29.089163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f204572-aa8e-455a-ae64-006b8de40980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:29.089985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f204572-aa8e-455a-ae64-006b8de40980 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:29.090274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f204572-aa8e-455a-ae64-006b8de40980 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:29.090552 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f204572-aa8e-455a-ae64-006b8de40980 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:29.099573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f204572-aa8e-455a-ae64-006b8de40980 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:29.101060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f204572-aa8e-455a-ae64-006b8de40980 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:29.116313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-6f204572-aa8e-455a-ae64-006b8de40980 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:29.116313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:29.116313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:29.116313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:29.116313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:29.116313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:29.116313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:29.116313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:29.116313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:29.116313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:29.116313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:29.116313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:29.116313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:29.116313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:29.116313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:29.117339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:29.118245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:29.118245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:29.118245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:29.118245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:29.118245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:29.118245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:29.118245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:29.118245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:29.118245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 411/830] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:29.127778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823f5c5-70cd-443a-af87-56d937965e36 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:29.234844 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823f5c5-70cd-443a-af87-56d937965e36 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:29.271463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f53d0-429a-4dba-b02b-ad996fbc137f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:29.272130 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f53d0-429a-4dba-b02b-ad996fbc137f None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:29.272433 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f53d0-429a-4dba-b02b-ad996fbc137f None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:29.272698 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f53d0-429a-4dba-b02b-ad996fbc137f None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:29.273501 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f53d0-429a-4dba-b02b-ad996fbc137f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:29.273815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f53d0-429a-4dba-b02b-ad996fbc137f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:29.275288 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f53d0-429a-4dba-b02b-ad996fbc137f None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:29.288106 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 420/831] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:29.293799 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f204572-aa8e-455a-ae64-006b8de40980 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:29.402168 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f204572-aa8e-455a-ae64-006b8de40980 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:29.423697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:29.438245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bf865-059f-45ba-8e0f-d72dd86041f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:29.438945 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bf865-059f-45ba-8e0f-d72dd86041f2 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:29.439399 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bf865-059f-45ba-8e0f-d72dd86041f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:29.439744 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bf865-059f-45ba-8e0f-d72dd86041f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:29.440812 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bf865-059f-45ba-8e0f-d72dd86041f2 None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=a498284274454d25b1b118a88996e522, user.id=be08c43f0d0c4a8194596e54500b44ed) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:29.441124 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bf865-059f-45ba-8e0f-d72dd86041f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:29.442552 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bf865-059f-45ba-8e0f-d72dd86041f2 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:29.449408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 412/832] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:00:29 2022] GET /identity/v3/role_assignments?user.id=be08c43f0d0c4a8194596e54500b44ed&scope.project.id=a498284274454d25b1b118a88996e522&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:29.764564 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb299a8-20f6-4de5-b65b-f46a2cf00c8c None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:29.765068 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb299a8-20f6-4de5-b65b-f46a2cf00c8c None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:29.765462 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb299a8-20f6-4de5-b65b-f46a2cf00c8c None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:29.767247 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 421/833] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:29.772682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18ad8f-ba45-48b5-881a-aa9d42967281 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:29.773134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18ad8f-ba45-48b5-881a-aa9d42967281 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:29.773516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18ad8f-ba45-48b5-881a-aa9d42967281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:29.790275 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-8b18ad8f-ba45-48b5-881a-aa9d42967281 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:29.791482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8b18ad8f-ba45-48b5-881a-aa9d42967281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:29.865104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 413/834] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:00:29.885389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb299a8-20f6-4de5-b65b-f46a2cf00c8c None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:29.906289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:29.992791 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d511d30-8712-4cc8-8fe3-158c1bb53851 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:29.993123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d511d30-8712-4cc8-8fe3-158c1bb53851 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:29.993362 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d511d30-8712-4cc8-8fe3-158c1bb53851 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:29.994108 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb299a8-20f6-4de5-b65b-f46a2cf00c8c None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:29.995159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 414/835] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:30.000954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1216e-2dba-4d97-9c6e-18ae0c57cc96 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:30.001240 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1216e-2dba-4d97-9c6e-18ae0c57cc96 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:30.001520 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1216e-2dba-4d97-9c6e-18ae0c57cc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:30.018191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-7bc1216e-2dba-4d97-9c6e-18ae0c57cc96 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:30.019405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7bc1216e-2dba-4d97-9c6e-18ae0c57cc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:30.040979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b1bf9-20f5-4dc2-a7ea-878d020f0b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:30.042054 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b1bf9-20f5-4dc2-a7ea-878d020f0b69 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:30.043011 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b1bf9-20f5-4dc2-a7ea-878d020f0b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:30.043437 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b1bf9-20f5-4dc2-a7ea-878d020f0b69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:30.060751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b1bf9-20f5-4dc2-a7ea-878d020f0b69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:30.070123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-cb3b1bf9-20f5-4dc2-a7ea-878d020f0b69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:00:30.070123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:30.070123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:30.070123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:30.070123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:30.070123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:30.070123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:30.070123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:30.070123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:30.070123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:30.070123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:30.070123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:30.070123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:30.070123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:30.070123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:30.070123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:30.071087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:30.071087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:30.071087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:30.071087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:30.071087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:30.071087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:30.071087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:30.071087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:30.071087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:30.071087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:30.071087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:30.071087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:30.071087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:30.071087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:30.071087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:30.071087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:30.071940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:30.071940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:30.071940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:30.071940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:30.071940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:30.071940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:30.071940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:30.071940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:30.071940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:30.071940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:30.071940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:00:30.071940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:30.073394 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 422/836] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:00:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:30.074839 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7c4bb-4201-4cbe-b001-8758a29baacd None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:30.075263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7c4bb-4201-4cbe-b001-8758a29baacd None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:30.076107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7c4bb-4201-4cbe-b001-8758a29baacd None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:30.078803 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 423/837] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:30.079566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7c4bb-4201-4cbe-b001-8758a29baacd None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:30.104265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 415/838] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:00:30.106759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576405be-0c76-4079-8bfb-caa975939930 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:30.107588 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576405be-0c76-4079-8bfb-caa975939930 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:30.108179 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576405be-0c76-4079-8bfb-caa975939930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:30.129262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-576405be-0c76-4079-8bfb-caa975939930 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:30.131131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-576405be-0c76-4079-8bfb-caa975939930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:30.166242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:30.174930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:30.193955 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7c4bb-4201-4cbe-b001-8758a29baacd None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:30.208824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 416/839] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:30.219560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576405be-0c76-4079-8bfb-caa975939930 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:30.230208 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741106ac-3135-40bc-9cd0-fcfc3f7b5f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:30.230995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741106ac-3135-40bc-9cd0-fcfc3f7b5f5f None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:30.231383 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741106ac-3135-40bc-9cd0-fcfc3f7b5f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:30.231852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741106ac-3135-40bc-9cd0-fcfc3f7b5f5f None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:30.232914 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741106ac-3135-40bc-9cd0-fcfc3f7b5f5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:30.233226 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741106ac-3135-40bc-9cd0-fcfc3f7b5f5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:30.234646 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741106ac-3135-40bc-9cd0-fcfc3f7b5f5f None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:30.248567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 424/840] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:00:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:30.249374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741106ac-3135-40bc-9cd0-fcfc3f7b5f5f None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:30.361433 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741106ac-3135-40bc-9cd0-fcfc3f7b5f5f None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:30.397520 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1c10f-0585-4ac6-891c-b6aaf9c3f4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:30.398226 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1c10f-0585-4ac6-891c-b6aaf9c3f4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:30.398510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1c10f-0585-4ac6-891c-b6aaf9c3f4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:30.398808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1c10f-0585-4ac6-891c-b6aaf9c3f4d8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:30.399456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1c10f-0585-4ac6-891c-b6aaf9c3f4d8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:30.400816 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1c10f-0585-4ac6-891c-b6aaf9c3f4d8 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-e5e1c10f-0585-4ac6-891c-b6aaf9c3f4d8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:30.412612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:30.413613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:30.413613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:30.413613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:30.413613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:30.413613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:30.413613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:30.413613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:30.413613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:30.413613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:30.413613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:30.413613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:30.413613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:30.413613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:30.413613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:30.413613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:30.413613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:30.414460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:30.414460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:30.414460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:30.414460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:30.414460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:30.414460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:30.414460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:30.414460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:30.414460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:30.414460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:30.414460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:30.414460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:30.414460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:00:30.414460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:30.414460 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 425/841] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 09:00:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:30.415807 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1c10f-0585-4ac6-891c-b6aaf9c3f4d8 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:30.418176 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:30.449116 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576405be-0c76-4079-8bfb-caa975939930 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:30.484999 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074f879-ddab-42e5-8cc8-eac785e967cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:30.485789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074f879-ddab-42e5-8cc8-eac785e967cd None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:30.486069 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074f879-ddab-42e5-8cc8-eac785e967cd None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:30.486347 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074f879-ddab-42e5-8cc8-eac785e967cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:30.496462 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074f879-ddab-42e5-8cc8-eac785e967cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:30.504663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-8074f879-ddab-42e5-8cc8-eac785e967cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:30.504663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:30.504663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:30.504663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:30.504663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:30.504663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:30.504663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:30.504663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:30.504663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:30.504663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:30.504663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:30.504663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:30.504663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:30.504663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:30.504663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:30.504663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:30.505689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:30.505689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:30.505689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:30.505689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:30.505689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:30.505689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:30.505689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:30.505689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:30.505689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:30.505689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:30.505689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:30.505689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:30.505689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:30.505689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:30.505689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:30.505689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:30.506542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:30.506542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:30.506542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:30.506542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:30.506542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:30.506542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:30.506542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:30.506542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:30.506542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:30.506542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:30.506542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:00:30.506542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:30.506542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 417/842] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:00:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:30.507261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074f879-ddab-42e5-8cc8-eac785e967cd None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:30.529929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1c10f-0585-4ac6-891c-b6aaf9c3f4d8 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:30.545827 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:30.569947 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ab378-2ea3-4d22-a18a-4780bd25491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:30.570781 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ab378-2ea3-4d22-a18a-4780bd25491f None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:30.571076 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ab378-2ea3-4d22-a18a-4780bd25491f None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:30.571358 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ab378-2ea3-4d22-a18a-4780bd25491f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:30.582348 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ab378-2ea3-4d22-a18a-4780bd25491f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:30.583840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ab378-2ea3-4d22-a18a-4780bd25491f None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:30.600659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-474ab378-2ea3-4d22-a18a-4780bd25491f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:30.600659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:30.600659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:30.600659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:30.600659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:30.600659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:30.600659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:30.600659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:30.600659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:30.600659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:30.600659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:30.600659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:30.600659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:30.600659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:30.600659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:30.601750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:30.602664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:30.602664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:30.602664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:30.602664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:30.602664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 426/843] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:30.602664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ab378-2ea3-4d22-a18a-4780bd25491f None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:30.627830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074f879-ddab-42e5-8cc8-eac785e967cd None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:30.665923 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed19e0-cda3-43c9-990b-24bb90f9a388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:30.666537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed19e0-cda3-43c9-990b-24bb90f9a388 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:30.666851 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed19e0-cda3-43c9-990b-24bb90f9a388 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:30.667125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed19e0-cda3-43c9-990b-24bb90f9a388 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:30.667833 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed19e0-cda3-43c9-990b-24bb90f9a388 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:30.668034 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:30.668139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed19e0-cda3-43c9-990b-24bb90f9a388 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:30.670271 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed19e0-cda3-43c9-990b-24bb90f9a388 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:30.675440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 418/844] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 09:00:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:30.676169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed19e0-cda3-43c9-990b-24bb90f9a388 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:30.725906 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ab378-2ea3-4d22-a18a-4780bd25491f None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:30.761979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ce85f-755d-4ce7-b790-a415e965bbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:30.763449 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ce85f-755d-4ce7-b790-a415e965bbc7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:30.763825 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ce85f-755d-4ce7-b790-a415e965bbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:30.764241 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ce85f-755d-4ce7-b790-a415e965bbc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:30.764864 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ce85f-755d-4ce7-b790-a415e965bbc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:30.765177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ce85f-755d-4ce7-b790-a415e965bbc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:30.766580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ce85f-755d-4ce7-b790-a415e965bbc7 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:30.779807 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 427/845] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 09:00:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:30.779807 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ce85f-755d-4ce7-b790-a415e965bbc7 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:30.787715 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:30.793202 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed19e0-cda3-43c9-990b-24bb90f9a388 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:30.830928 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65839fc7-f6a0-4b1e-bace-b04e180b477e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:30.831683 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65839fc7-f6a0-4b1e-bace-b04e180b477e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:30.831972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65839fc7-f6a0-4b1e-bace-b04e180b477e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:30.832250 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65839fc7-f6a0-4b1e-bace-b04e180b477e None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:30.832970 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65839fc7-f6a0-4b1e-bace-b04e180b477e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:30.833276 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65839fc7-f6a0-4b1e-bace-b04e180b477e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:30.834643 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65839fc7-f6a0-4b1e-bace-b04e180b477e None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:30.845726 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 419/846] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:30.846515 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65839fc7-f6a0-4b1e-bace-b04e180b477e None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:30.890091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ce85f-755d-4ce7-b790-a415e965bbc7 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:30.908279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:30.926862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52246a3b-db4d-4eff-af7d-2f59ab771fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:30.927457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52246a3b-db4d-4eff-af7d-2f59ab771fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:30.927744 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52246a3b-db4d-4eff-af7d-2f59ab771fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:30.928068 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52246a3b-db4d-4eff-af7d-2f59ab771fe2 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:30.928727 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52246a3b-db4d-4eff-af7d-2f59ab771fe2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:30.929035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52246a3b-db4d-4eff-af7d-2f59ab771fe2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:30.930400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52246a3b-db4d-4eff-af7d-2f59ab771fe2 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:30.942719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 428/847] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 09:00:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:30.942719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52246a3b-db4d-4eff-af7d-2f59ab771fe2 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:30.964458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65839fc7-f6a0-4b1e-bace-b04e180b477e None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:31.005894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad181e-3d1e-42e4-b0b3-3f349e17c17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:31.006767 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad181e-3d1e-42e4-b0b3-3f349e17c17a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:31.007055 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad181e-3d1e-42e4-b0b3-3f349e17c17a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:31.007336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad181e-3d1e-42e4-b0b3-3f349e17c17a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:31.020310 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad181e-3d1e-42e4-b0b3-3f349e17c17a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:31.021837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad181e-3d1e-42e4-b0b3-3f349e17c17a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:31.041452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-e1ad181e-3d1e-42e4-b0b3-3f349e17c17a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:31.041452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:31.041452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:31.041452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:31.041452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:31.041452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:31.041452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:31.041452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:31.041452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:31.041452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:31.041452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:31.041452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:31.041452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:31.041452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:31.041452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:31.042423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:31.043332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:31.043332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:31.043332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:31.043332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:31.047916 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 420/848] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:31.049363 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad181e-3d1e-42e4-b0b3-3f349e17c17a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:31.063565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:31.072677 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52246a3b-db4d-4eff-af7d-2f59ab771fe2 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:31.118399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93bf58-2fe4-43a0-ace3-d5fb766fc917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:31.119564 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93bf58-2fe4-43a0-ace3-d5fb766fc917 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:31.119857 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93bf58-2fe4-43a0-ace3-d5fb766fc917 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:31.120354 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93bf58-2fe4-43a0-ace3-d5fb766fc917 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:31.130003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93bf58-2fe4-43a0-ace3-d5fb766fc917 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:31.132131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93bf58-2fe4-43a0-ace3-d5fb766fc917 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:31.149972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-5a93bf58-2fe4-43a0-ace3-d5fb766fc917 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:00:31.149972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:31.149972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:31.149972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:31.149972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:31.149972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:31.149972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:31.149972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:31.149972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:31.149972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:31.149972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:31.149972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:31.149972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:31.149972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:31.149972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:31.149972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:31.151058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:31.152001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:31.152001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:31.152001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:31.152001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:31.152001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:31.152001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:31.152001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:31.152001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:31.152001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 09:00:31.152001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:31.152001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 429/849] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:00:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:31.158806 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93bf58-2fe4-43a0-ace3-d5fb766fc917 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:31.173504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad181e-3d1e-42e4-b0b3-3f349e17c17a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:31.214962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ba6ba-7a1a-4156-9301-b316281a0726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:31.215576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:31.217582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba6ba-7a1a-4156-9301-b316281a0726 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:31.218201 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba6ba-7a1a-4156-9301-b316281a0726 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:31.218494 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba6ba-7a1a-4156-9301-b316281a0726 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:31.219423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ba6ba-7a1a-4156-9301-b316281a0726 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:31.219959 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ba6ba-7a1a-4156-9301-b316281a0726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:31.221956 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ba6ba-7a1a-4156-9301-b316281a0726 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:31.242782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 421/850] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:31.244569 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ba6ba-7a1a-4156-9301-b316281a0726 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:31.295728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93bf58-2fe4-43a0-ace3-d5fb766fc917 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:31.339403 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d20e11-e1a9-4c3b-af28-ee17afca63d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:31.340094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d20e11-e1a9-4c3b-af28-ee17afca63d0 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:31.340377 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d20e11-e1a9-4c3b-af28-ee17afca63d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:31.340696 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d20e11-e1a9-4c3b-af28-ee17afca63d0 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:31.342162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d20e11-e1a9-4c3b-af28-ee17afca63d0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:31.342858 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d20e11-e1a9-4c3b-af28-ee17afca63d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:31.344635 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d20e11-e1a9-4c3b-af28-ee17afca63d0 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:31.351056 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:31.367332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 430/851] 10.222.0.44 () {56 vars in 1135 bytes} [Wed Jul 20 09:00:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:31.369768 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ba6ba-7a1a-4156-9301-b316281a0726 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:31.374743 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d20e11-e1a9-4c3b-af28-ee17afca63d0 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:31.415928 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017ab38-5c29-4e8c-b24d-f3d8a9e792dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:31.417272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017ab38-5c29-4e8c-b24d-f3d8a9e792dd None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:31.417688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017ab38-5c29-4e8c-b24d-f3d8a9e792dd None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:31.417976 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017ab38-5c29-4e8c-b24d-f3d8a9e792dd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:31.427548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017ab38-5c29-4e8c-b24d-f3d8a9e792dd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:31.428982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017ab38-5c29-4e8c-b24d-f3d8a9e792dd None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:31.450847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-3017ab38-5c29-4e8c-b24d-f3d8a9e792dd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:00:31.450847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:31.450847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:31.450847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:31.450847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:31.450847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:31.450847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:31.450847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:31.450847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:31.450847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:31.450847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:31.450847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:31.450847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:31.450847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:31.450847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:31.450847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:31.452641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:31.454731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:31.454731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:31.454731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:31.454731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:31.454731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:31.454731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:31.454731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:31.454731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:31.454731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:00:31.454731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:31.454731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 422/852] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jul 20 09:00:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:31.459599 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017ab38-5c29-4e8c-b24d-f3d8a9e792dd None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:31.509527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:31.514309 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d20e11-e1a9-4c3b-af28-ee17afca63d0 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:31.563846 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff34a2-a40d-4687-97eb-1a175621b7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:31.564793 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff34a2-a40d-4687-97eb-1a175621b7ad None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:31.565385 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff34a2-a40d-4687-97eb-1a175621b7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:31.565866 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff34a2-a40d-4687-97eb-1a175621b7ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:31.580721 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017ab38-5c29-4e8c-b24d-f3d8a9e792dd None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:31.613306 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:31.620180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c1ec8-ca1b-47f0-94a8-33a7271a11cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:31.620989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c1ec8-ca1b-47f0-94a8-33a7271a11cd None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:31.621281 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c1ec8-ca1b-47f0-94a8-33a7271a11cd None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:31.622503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c1ec8-ca1b-47f0-94a8-33a7271a11cd None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:31.623412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c1ec8-ca1b-47f0-94a8-33a7271a11cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:31.623876 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c1ec8-ca1b-47f0-94a8-33a7271a11cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:31.625433 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c1ec8-ca1b-47f0-94a8-33a7271a11cd None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:31.637097 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff34a2-a40d-4687-97eb-1a175621b7ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:31.638856 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff34a2-a40d-4687-97eb-1a175621b7ad None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:31.644215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 423/853] 10.222.0.44 () {56 vars in 1149 bytes} [Wed Jul 20 09:00:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:31.645470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654a637-8131-45ff-9d4d-8b6c9cfd56cf None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:31.645765 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654a637-8131-45ff-9d4d-8b6c9cfd56cf None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:31.646042 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654a637-8131-45ff-9d4d-8b6c9cfd56cf None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:31.647578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 424/854] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:31.650797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654a637-8131-45ff-9d4d-8b6c9cfd56cf None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:31.743741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:31.752657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 431/855] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:00:31 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 09:00:31.754038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7da3e0-0140-444a-925e-d7e3ddc09d8a None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:31.754254 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7da3e0-0140-444a-925e-d7e3ddc09d8a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:31.754540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7da3e0-0140-444a-925e-d7e3ddc09d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:31.758854 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654a637-8131-45ff-9d4d-8b6c9cfd56cf None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:31.772658 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-8e7da3e0-0140-444a-925e-d7e3ddc09d8a None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:31.773784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8e7da3e0-0140-444a-925e-d7e3ddc09d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:31.795690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01149cc3-9fb5-4131-a5c7-0e506b555336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:31.796347 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01149cc3-9fb5-4131-a5c7-0e506b555336 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:31.796635 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01149cc3-9fb5-4131-a5c7-0e506b555336 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:31.796915 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01149cc3-9fb5-4131-a5c7-0e506b555336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:31.847001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 432/856] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:31.848113 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7da3e0-0140-444a-925e-d7e3ddc09d8a None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:31.854614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:31.869209 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01149cc3-9fb5-4131-a5c7-0e506b555336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:31.871135 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01149cc3-9fb5-4131-a5c7-0e506b555336 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:31.963194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7da3e0-0140-444a-925e-d7e3ddc09d8a None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:31.976909 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 425/857] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:00:31 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:00:31.977578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01149cc3-9fb5-4131-a5c7-0e506b555336 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:32.000836 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:32.007468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cb948-c003-44f9-81a0-29c52932b6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:32.008264 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cb948-c003-44f9-81a0-29c52932b6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:32.008552 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cb948-c003-44f9-81a0-29c52932b6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:32.008987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cb948-c003-44f9-81a0-29c52932b6a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:32.020495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cb948-c003-44f9-81a0-29c52932b6a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:32.021896 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cb948-c003-44f9-81a0-29c52932b6a4 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:32.040686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-fc3cb948-c003-44f9-81a0-29c52932b6a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:32.040686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:32.040686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:32.040686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:32.040686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:32.040686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:32.040686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:32.040686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:32.040686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:32.040686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:32.040686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:32.040686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:32.040686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:32.040686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:32.040686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:32.041576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:32.042468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:32.042468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:32.042468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:32.042468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:32.042468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 433/858] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:32.043256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cb948-c003-44f9-81a0-29c52932b6a4 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:32.069782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:32.083769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01149cc3-9fb5-4131-a5c7-0e506b555336 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:32.121000 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db238ee-1cef-4f34-8a09-dd5279441c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:32.122312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db238ee-1cef-4f34-8a09-dd5279441c1b None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:32.122594 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db238ee-1cef-4f34-8a09-dd5279441c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:32.122980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db238ee-1cef-4f34-8a09-dd5279441c1b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:32.123745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db238ee-1cef-4f34-8a09-dd5279441c1b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:32.125927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db238ee-1cef-4f34-8a09-dd5279441c1b None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:32.141495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-3db238ee-1cef-4f34-8a09-dd5279441c1b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:00:32.141495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:32.141495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:32.141495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:32.141495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:32.141495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:32.141495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:32.141495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:32.141495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:00:32.141495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:32.141495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:32.141495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:32.141495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:32.141495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:32.141495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:32.141495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:32.142466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:32.142466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:32.142466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:32.142466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:32.142466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:32.142466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:32.142466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:32.142466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:32.142466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:32.142466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:32.142466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:32.142466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:32.142466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:32.142466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:32.142466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:32.142466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:32.143346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:32.143346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:32.143346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:32.143346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:32.143346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:32.143346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:32.143346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:32.143346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:32.143346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:32.143346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:32.143346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:32.143346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:32.143346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:32.143346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:00:32.143346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:32.144143 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 426/859] 10.222.0.44 () {56 vars in 1132 bytes} [Wed Jul 20 09:00:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:32.144819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db238ee-1cef-4f34-8a09-dd5279441c1b None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:32.158732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cb948-c003-44f9-81a0-29c52932b6a4 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:32.195822 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d759e5-dca6-44b2-8d50-7d93d5e0c9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:32.196618 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d759e5-dca6-44b2-8d50-7d93d5e0c9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:32.196932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d759e5-dca6-44b2-8d50-7d93d5e0c9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:32.197182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d759e5-dca6-44b2-8d50-7d93d5e0c9d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:32.210467 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d759e5-dca6-44b2-8d50-7d93d5e0c9d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:32.211971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d759e5-dca6-44b2-8d50-7d93d5e0c9d3 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:32.230436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-a3d759e5-dca6-44b2-8d50-7d93d5e0c9d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:32.230436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:32.230436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:32.230436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:32.230436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:32.230436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:32.230436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:32.230436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:32.230436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:32.230436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:32.230436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:32.230436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:32.230436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:32.230436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:32.230436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:32.231881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:32.233063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:32.233063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:32.233063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:32.233063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:32.233063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 434/860] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:32.233063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d759e5-dca6-44b2-8d50-7d93d5e0c9d3 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:32.256646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db238ee-1cef-4f34-8a09-dd5279441c1b None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:32.277187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:32.293221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e218b21-c2e4-460e-8d1b-e0624802fa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:32.293953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e218b21-c2e4-460e-8d1b-e0624802fa36 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:32.294429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e218b21-c2e4-460e-8d1b-e0624802fa36 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:32.294818 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e218b21-c2e4-460e-8d1b-e0624802fa36 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:32.295550 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e218b21-c2e4-460e-8d1b-e0624802fa36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:32.295854 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e218b21-c2e4-460e-8d1b-e0624802fa36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:32.297216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e218b21-c2e4-460e-8d1b-e0624802fa36 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:32.315375 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 427/861] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:32.315375 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e218b21-c2e4-460e-8d1b-e0624802fa36 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:32.333741 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:32.347674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d759e5-dca6-44b2-8d50-7d93d5e0c9d3 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:32.386412 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ced22-972d-4a74-b3e5-01b9a82057df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:32.387129 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ced22-972d-4a74-b3e5-01b9a82057df None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:32.387420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ced22-972d-4a74-b3e5-01b9a82057df None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:32.388327 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ced22-972d-4a74-b3e5-01b9a82057df None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:32.389038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ced22-972d-4a74-b3e5-01b9a82057df None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:32.389350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ced22-972d-4a74-b3e5-01b9a82057df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:32.390736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ced22-972d-4a74-b3e5-01b9a82057df None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:32.399751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 435/862] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jul 20 09:00:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:32.399751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ced22-972d-4a74-b3e5-01b9a82057df None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:32.428152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e218b21-c2e4-460e-8d1b-e0624802fa36 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:32.464616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2545e-53bb-4d8e-b830-fae7fcbff3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:32.465574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2545e-53bb-4d8e-b830-fae7fcbff3ae None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:32.466410 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2545e-53bb-4d8e-b830-fae7fcbff3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:32.466713 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2545e-53bb-4d8e-b830-fae7fcbff3ae None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:32.467446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2545e-53bb-4d8e-b830-fae7fcbff3ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:32.467760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2545e-53bb-4d8e-b830-fae7fcbff3ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:32.469124 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2545e-53bb-4d8e-b830-fae7fcbff3ae None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:32.483205 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 428/863] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:32.484005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2545e-53bb-4d8e-b830-fae7fcbff3ae None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:32.504018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:32.507927 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ced22-972d-4a74-b3e5-01b9a82057df None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:32.551959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce4be4-024f-4365-babf-175535acc214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:32.552950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce4be4-024f-4365-babf-175535acc214 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:32.553245 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce4be4-024f-4365-babf-175535acc214 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:32.553679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce4be4-024f-4365-babf-175535acc214 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:32.561461 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce4be4-024f-4365-babf-175535acc214 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:32.562944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce4be4-024f-4365-babf-175535acc214 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:32.563644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:32.578637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-9bce4be4-024f-4365-babf-175535acc214 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:32.578637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:32.578637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:32.578637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:32.578637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:32.578637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:32.578637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:32.578637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:32.578637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:32.578637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:32.578637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:32.578637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:32.578637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:32.578637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:32.578637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:32.579588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:32.580491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:32.580491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:32.580491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:32.580491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:32.580491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:32.580491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:32.580491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:32.580491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:32.580491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 436/864] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:32.581396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce4be4-024f-4365-babf-175535acc214 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:32.603724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2545e-53bb-4d8e-b830-fae7fcbff3ae None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:32.641770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03960ccc-1928-48fe-9723-5c06df483190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:32.642438 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03960ccc-1928-48fe-9723-5c06df483190 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:32.642755 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03960ccc-1928-48fe-9723-5c06df483190 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:32.643039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03960ccc-1928-48fe-9723-5c06df483190 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:32.643750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03960ccc-1928-48fe-9723-5c06df483190 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:32.644058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03960ccc-1928-48fe-9723-5c06df483190 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:32.645429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03960ccc-1928-48fe-9723-5c06df483190 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:32.649931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 429/865] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jul 20 09:00:32 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:32.650825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03960ccc-1928-48fe-9723-5c06df483190 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:32.712958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce4be4-024f-4365-babf-175535acc214 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:32.763920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971babf7-0052-4a71-b11d-00a141067a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:32.765830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971babf7-0052-4a71-b11d-00a141067a13 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:32.766343 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971babf7-0052-4a71-b11d-00a141067a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:32.767097 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971babf7-0052-4a71-b11d-00a141067a13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:32.778016 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971babf7-0052-4a71-b11d-00a141067a13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:32.779147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03960ccc-1928-48fe-9723-5c06df483190 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:32.780104 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971babf7-0052-4a71-b11d-00a141067a13 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:32.801549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-971babf7-0052-4a71-b11d-00a141067a13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:32.801549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:32.801549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:32.801549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:32.801549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:32.801549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:32.801549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:32.801549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:32.801549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:32.801549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:32.801549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:32.801549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:32.801549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:32.801549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:32.801549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:32.802556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:32.803521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:32.803521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:32.803521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:32.803521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:32.803521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:32.803521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:32.803521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:32.803521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:32.810602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 437/866] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:32.811790 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971babf7-0052-4a71-b11d-00a141067a13 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:32.827455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe489d84-191d-410d-a90f-3fd3b74e6a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:32.828249 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe489d84-191d-410d-a90f-3fd3b74e6a8c None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:32.828536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe489d84-191d-410d-a90f-3fd3b74e6a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:32.828815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe489d84-191d-410d-a90f-3fd3b74e6a8c None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:32.829614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe489d84-191d-410d-a90f-3fd3b74e6a8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:32.829923 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe489d84-191d-410d-a90f-3fd3b74e6a8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:32.831536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe489d84-191d-410d-a90f-3fd3b74e6a8c None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:32.835193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:32.843530 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:32.847711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 430/867] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:32.854327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe489d84-191d-410d-a90f-3fd3b74e6a8c None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:32.945290 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971babf7-0052-4a71-b11d-00a141067a13 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:32.988283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685dbe1d-5612-4792-bb79-610e67e3e7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:32.988948 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685dbe1d-5612-4792-bb79-610e67e3e7c0 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:32.989232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685dbe1d-5612-4792-bb79-610e67e3e7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:32.989507 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685dbe1d-5612-4792-bb79-610e67e3e7c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:32.990295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685dbe1d-5612-4792-bb79-610e67e3e7c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:32.990602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685dbe1d-5612-4792-bb79-610e67e3e7c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:32.992057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685dbe1d-5612-4792-bb79-610e67e3e7c0 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:32.999541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe489d84-191d-410d-a90f-3fd3b74e6a8c None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:33.011792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 438/868] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:33.018415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685dbe1d-5612-4792-bb79-610e67e3e7c0 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:33.039169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4cfcf6-439c-4dff-b34f-f4f5fc9ecd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:33.039970 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cfcf6-439c-4dff-b34f-f4f5fc9ecd81 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:33.040355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cfcf6-439c-4dff-b34f-f4f5fc9ecd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:33.040733 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4cfcf6-439c-4dff-b34f-f4f5fc9ecd81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:33.041693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cfcf6-439c-4dff-b34f-f4f5fc9ecd81 None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=a498284274454d25b1b118a88996e522, user.id=03b4375e6ead42fb8d77a7b30617cb95) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:33.042104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cfcf6-439c-4dff-b34f-f4f5fc9ecd81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:33.043657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4cfcf6-439c-4dff-b34f-f4f5fc9ecd81 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:33.054788 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 431/869] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:00:32 2022] GET /identity/v3/role_assignments?user.id=03b4375e6ead42fb8d77a7b30617cb95&scope.project.id=a498284274454d25b1b118a88996e522&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:33.112624 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:33.139005 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685dbe1d-5612-4792-bb79-610e67e3e7c0 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:33.149620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d366152-4a47-4542-8992-d2eb9da977a3 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:33.150065 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d366152-4a47-4542-8992-d2eb9da977a3 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:33.150450 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d366152-4a47-4542-8992-d2eb9da977a3 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:33.153443 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 432/870] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:33.158308 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d4632-191e-4daf-9fd6-bc7c1626c6a1 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:33.158773 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d4632-191e-4daf-9fd6-bc7c1626c6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:33.159158 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d4632-191e-4daf-9fd6-bc7c1626c6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:33.177588 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-337d4632-191e-4daf-9fd6-bc7c1626c6a1 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:33.177794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b1a17-343b-4d94-8041-c6d13a48b8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:33.178325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b1a17-343b-4d94-8041-c6d13a48b8ba None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:33.178615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b1a17-343b-4d94-8041-c6d13a48b8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:33.178929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b1a17-343b-4d94-8041-c6d13a48b8ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:33.179049 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-337d4632-191e-4daf-9fd6-bc7c1626c6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:33.179778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b1a17-343b-4d94-8041-c6d13a48b8ba None admin] RBAC: Adding query filter params (role.id=ae949d8bea4b40d9b1a9fa207d539663, scope.project.id=a498284274454d25b1b118a88996e522, user.id=6b8db4a7ee284b1aa96d4517dd381e71) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:33.180096 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b1a17-343b-4d94-8041-c6d13a48b8ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:33.181564 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b1a17-343b-4d94-8041-c6d13a48b8ba None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:33.189445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 439/871] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:00:33 2022] GET /identity/v3/role_assignments?user.id=6b8db4a7ee284b1aa96d4517dd381e71&scope.project.id=a498284274454d25b1b118a88996e522&role.id=ae949d8bea4b40d9b1a9fa207d539663 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:33.249259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:33.269149 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 433/872] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:00:33.292441 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b1a17-343b-4d94-8041-c6d13a48b8ba None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:33.414850 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b1a17-343b-4d94-8041-c6d13a48b8ba None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:33.452363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0738ba8-b881-438d-bbe3-20c6f4e87dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:33.453041 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0738ba8-b881-438d-bbe3-20c6f4e87dea None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:33.453325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0738ba8-b881-438d-bbe3-20c6f4e87dea None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:33.453604 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0738ba8-b881-438d-bbe3-20c6f4e87dea None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:33.454332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0738ba8-b881-438d-bbe3-20c6f4e87dea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:33.455753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0738ba8-b881-438d-bbe3-20c6f4e87dea None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:33.471808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 440/873] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jul 20 09:00:33 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:00:34.778805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a559917-2b9d-492c-ad90-c14b10694eeb None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:34.779416 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a559917-2b9d-492c-ad90-c14b10694eeb None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:34.779416 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a559917-2b9d-492c-ad90-c14b10694eeb None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:34.780952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 434/874] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:34.786297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcff99e-8439-4930-9861-cf040269cff1 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:34.786719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcff99e-8439-4930-9861-cf040269cff1 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:34.787011 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcff99e-8439-4930-9861-cf040269cff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:34.792409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:34.808083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-ebcff99e-8439-4930-9861-cf040269cff1 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:34.809204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ebcff99e-8439-4930-9861-cf040269cff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:34.885190 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 441/875] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:34.907618 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a559917-2b9d-492c-ad90-c14b10694eeb None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:35.012183 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a559917-2b9d-492c-ad90-c14b10694eeb None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:35.049881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811e2a9-d724-4dba-9132-0ecc3008bffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:35.050547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811e2a9-d724-4dba-9132-0ecc3008bffc None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:35.050861 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811e2a9-d724-4dba-9132-0ecc3008bffc None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:35.051140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811e2a9-d724-4dba-9132-0ecc3008bffc None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:35.051869 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811e2a9-d724-4dba-9132-0ecc3008bffc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:35.053234 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811e2a9-d724-4dba-9132-0ecc3008bffc None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:35.075630 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 435/876] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jul 20 09:00:34 2022] POST /identity/v3/services => generated 269 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:00:35.264313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74c944-57a7-4b5b-96a5-9814e7b73e39 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:35.264680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74c944-57a7-4b5b-96a5-9814e7b73e39 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:35.264881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74c944-57a7-4b5b-96a5-9814e7b73e39 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:35.266503 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 442/877] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:35.274955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dc689-9f20-4e06-9897-f3b3b045ad63 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:35.275480 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dc689-9f20-4e06-9897-f3b3b045ad63 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:35.275996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dc689-9f20-4e06-9897-f3b3b045ad63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:35.294629 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-2d4dc689-9f20-4e06-9897-f3b3b045ad63 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:35.295946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2d4dc689-9f20-4e06-9897-f3b3b045ad63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:35.347206 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:35.382904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 436/878] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:35.393312 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fcb1d-4f0f-44ec-a8ae-0880d425e294 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:35.393606 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fcb1d-4f0f-44ec-a8ae-0880d425e294 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:35.393891 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fcb1d-4f0f-44ec-a8ae-0880d425e294 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:35.395559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 443/879] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:35.400763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb61ee-8d30-48c2-a6b6-59fb4795e2c8 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:35.406718 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb61ee-8d30-48c2-a6b6-59fb4795e2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:35.406718 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb61ee-8d30-48c2-a6b6-59fb4795e2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:35.415364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fcb1d-4f0f-44ec-a8ae-0880d425e294 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:35.419448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-87eb61ee-8d30-48c2-a6b6-59fb4795e2c8 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:35.420956 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-87eb61ee-8d30-48c2-a6b6-59fb4795e2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:35.496337 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 437/880] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:35.520052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb61ee-8d30-48c2-a6b6-59fb4795e2c8 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:35.539136 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fcb1d-4f0f-44ec-a8ae-0880d425e294 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:35.581785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:35.593748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdf897-2adf-45f0-8dfc-d050c6a79624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:35.595408 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdf897-2adf-45f0-8dfc-d050c6a79624 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:35.596170 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdf897-2adf-45f0-8dfc-d050c6a79624 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:35.596639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdf897-2adf-45f0-8dfc-d050c6a79624 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:35.597925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbdf897-2adf-45f0-8dfc-d050c6a79624 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:35.600216 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbdf897-2adf-45f0-8dfc-d050c6a79624 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-0dbdf897-2adf-45f0-8dfc-d050c6a79624 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:35.618144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:35.619248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:35.619248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:35.619248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:35.619248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:35.619248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:35.619248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:35.619248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:35.619248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:35.619248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:35.619248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:35.619248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:35.619248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:35.619248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:35.619248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:35.619248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:35.619248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:35.620083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:35.620083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:35.620083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:35.620083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:35.620083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:35.620083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:35.620083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:35.620083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:35.620083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:35.620083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:35.620083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:35.620083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:35.620083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:00:35.620083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:35.623997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 444/881] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jul 20 09:00:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:35.634704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdf897-2adf-45f0-8dfc-d050c6a79624 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:35.651305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb61ee-8d30-48c2-a6b6-59fb4795e2c8 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:35.661105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:35.691077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ec3ce-accc-4a96-a534-c5f1cbe7b2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:35.691938 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ec3ce-accc-4a96-a534-c5f1cbe7b2ee None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:35.692248 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ec3ce-accc-4a96-a534-c5f1cbe7b2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:35.692642 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ec3ce-accc-4a96-a534-c5f1cbe7b2ee None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:35.704398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ec3ce-accc-4a96-a534-c5f1cbe7b2ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:35.712554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-290ec3ce-accc-4a96-a534-c5f1cbe7b2ee None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:00:35.712554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:35.712554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:35.712554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:35.712554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:35.712554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:35.712554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:35.712554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:35.712554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:35.712554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:35.712554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:35.712554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:35.712554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:35.712554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:35.712554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:35.712554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:35.713495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:35.713495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:35.713495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:35.713495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:35.713495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:35.713495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:35.713495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:35.713495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:35.713495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:35.713495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:35.713495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:35.713495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:35.713495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:35.713495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:35.713495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:35.713495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:35.714353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:35.714353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:35.714353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:35.714353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:35.714353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:35.714353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:35.714353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:35.714353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:35.714353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:35.714353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:35.714353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:00:35.714353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:35.714353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 438/882] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Jul 20 09:00:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:35.716966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2ee90-d9eb-44d4-8691-5d5d3563f825 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:35.716966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2ee90-d9eb-44d4-8691-5d5d3563f825 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:35.716966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2ee90-d9eb-44d4-8691-5d5d3563f825 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:35.717764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 439/883] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:35.721812 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2ee90-d9eb-44d4-8691-5d5d3563f825 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:35.744856 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdf897-2adf-45f0-8dfc-d050c6a79624 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:35.785650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b21893-acbc-4410-b6aa-ca1c2b81c053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:35.786326 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b21893-acbc-4410-b6aa-ca1c2b81c053 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:35.786945 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b21893-acbc-4410-b6aa-ca1c2b81c053 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:35.786945 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b21893-acbc-4410-b6aa-ca1c2b81c053 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:35.787601 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b21893-acbc-4410-b6aa-ca1c2b81c053 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:35.787911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b21893-acbc-4410-b6aa-ca1c2b81c053 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:35.789283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b21893-acbc-4410-b6aa-ca1c2b81c053 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:35.794323 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 445/884] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jul 20 09:00:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:35.795717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c0fc3-420a-4f67-93a6-5e4af854625c None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:35.796010 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c0fc3-420a-4f67-93a6-5e4af854625c None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:35.796289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c0fc3-420a-4f67-93a6-5e4af854625c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:35.813243 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-289c0fc3-420a-4f67-93a6-5e4af854625c None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:35.814427 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-289c0fc3-420a-4f67-93a6-5e4af854625c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:35.843779 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2ee90-d9eb-44d4-8691-5d5d3563f825 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:35.870659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:35.883951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d28b16-ce08-4f95-b128-0823cfd98403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:35.884628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d28b16-ce08-4f95-b128-0823cfd98403 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:35.884909 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d28b16-ce08-4f95-b128-0823cfd98403 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:35.885187 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d28b16-ce08-4f95-b128-0823cfd98403 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:35.885999 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d28b16-ce08-4f95-b128-0823cfd98403 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:35.886309 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d28b16-ce08-4f95-b128-0823cfd98403 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:35.888775 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d28b16-ce08-4f95-b128-0823cfd98403 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:35.894876 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 446/885] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:35.894876 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c0fc3-420a-4f67-93a6-5e4af854625c None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:35.909638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 440/886] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Jul 20 09:00:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:35.910773 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d28b16-ce08-4f95-b128-0823cfd98403 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:35.941920 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:36.006913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c0fc3-420a-4f67-93a6-5e4af854625c None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:36.018105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d28b16-ce08-4f95-b128-0823cfd98403 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:36.046696 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5731064-5db1-45cd-bbe6-6a1e119c6702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:36.047363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5731064-5db1-45cd-bbe6-6a1e119c6702 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:36.047641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5731064-5db1-45cd-bbe6-6a1e119c6702 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:36.047918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5731064-5db1-45cd-bbe6-6a1e119c6702 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:36.048629 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5731064-5db1-45cd-bbe6-6a1e119c6702 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:36.048944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5731064-5db1-45cd-bbe6-6a1e119c6702 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:36.050397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5731064-5db1-45cd-bbe6-6a1e119c6702 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:36.053235 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7c72f-cb3e-47db-9755-8481c5a6b22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:36.054022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7c72f-cb3e-47db-9755-8481c5a6b22c None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:36.054307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7c72f-cb3e-47db-9755-8481c5a6b22c None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:36.054585 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7c72f-cb3e-47db-9755-8481c5a6b22c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:36.055495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 447/887] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jul 20 09:00:35 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:36.056154 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5731064-5db1-45cd-bbe6-6a1e119c6702 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:36.078514 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7c72f-cb3e-47db-9755-8481c5a6b22c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:36.080542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7c72f-cb3e-47db-9755-8481c5a6b22c None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:36.109087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-a4b7c72f-cb3e-47db-9755-8481c5a6b22c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:36.109087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:36.109087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:36.109087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:36.109087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:36.109087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:36.109087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:36.109087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:36.109087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:36.109087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:36.109087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:36.109087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:36.109087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:36.109087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:36.109087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:36.110054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:36.110981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:36.110981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:36.110981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:36.110981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:36.117244 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 441/888] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:36.117906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7c72f-cb3e-47db-9755-8481c5a6b22c None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:36.123387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:36.196153 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5731064-5db1-45cd-bbe6-6a1e119c6702 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:36.216518 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:36.240285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7c72f-cb3e-47db-9755-8481c5a6b22c None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:36.251556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d65fb-2771-42aa-b872-003198d2b30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:36.252852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d65fb-2771-42aa-b872-003198d2b30e None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:36.253162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d65fb-2771-42aa-b872-003198d2b30e None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:36.253447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d65fb-2771-42aa-b872-003198d2b30e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:36.254413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d65fb-2771-42aa-b872-003198d2b30e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:36.256163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d65fb-2771-42aa-b872-003198d2b30e None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-f25d65fb-2771-42aa-b872-003198d2b30e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:36.274873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:36.275958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:36.275958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:36.275958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:36.275958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:36.275958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:36.275958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:36.275958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:36.275958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:36.275958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:36.275958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:36.275958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:36.275958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:36.275958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:36.275958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:36.275958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:36.275958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:36.276796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:36.276796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:36.276796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:36.276796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:36.276796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:36.276796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:36.276796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:36.276796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:36.276796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:36.276796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:36.276796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:36.276796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:36.276796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:00:36.276796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:36.277541 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 448/889] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 09:00:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:36.282875 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2010e-b807-4d1c-88ff-dd26e26a9df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:36.283566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2010e-b807-4d1c-88ff-dd26e26a9df0 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:36.283839 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d65fb-2771-42aa-b872-003198d2b30e None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:36.283955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2010e-b807-4d1c-88ff-dd26e26a9df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:36.284133 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2010e-b807-4d1c-88ff-dd26e26a9df0 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:36.284874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2010e-b807-4d1c-88ff-dd26e26a9df0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:36.285189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2010e-b807-4d1c-88ff-dd26e26a9df0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:36.286564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2010e-b807-4d1c-88ff-dd26e26a9df0 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:36.299686 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 442/890] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:36.304980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2010e-b807-4d1c-88ff-dd26e26a9df0 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:36.402449 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d65fb-2771-42aa-b872-003198d2b30e None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:36.417543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:36.440844 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc225b6-436c-41ad-8e9a-1304fe319e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:36.441897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc225b6-436c-41ad-8e9a-1304fe319e39 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:36.442385 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc225b6-436c-41ad-8e9a-1304fe319e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:36.442917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc225b6-436c-41ad-8e9a-1304fe319e39 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:36.443830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc225b6-436c-41ad-8e9a-1304fe319e39 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:36.444570 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2010e-b807-4d1c-88ff-dd26e26a9df0 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:36.444832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc225b6-436c-41ad-8e9a-1304fe319e39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:36.446407 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc225b6-436c-41ad-8e9a-1304fe319e39 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:36.451989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 449/891] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 09:00:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:36.459982 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc225b6-436c-41ad-8e9a-1304fe319e39 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:36.486159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1879f0-a304-4cdb-89ba-a279f7938371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:36.487339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1879f0-a304-4cdb-89ba-a279f7938371 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:36.487847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1879f0-a304-4cdb-89ba-a279f7938371 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:36.488340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1879f0-a304-4cdb-89ba-a279f7938371 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:36.496469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1879f0-a304-4cdb-89ba-a279f7938371 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:36.498782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1879f0-a304-4cdb-89ba-a279f7938371 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:36.501662 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:36.515989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-4d1879f0-a304-4cdb-89ba-a279f7938371 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:00:36.515989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:36.515989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:36.515989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:36.515989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:36.515989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:36.515989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:36.515989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:36.515989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:36.515989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:36.515989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:36.515989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:36.515989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:36.515989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:36.515989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:36.515989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:36.516962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:36.517894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:36.517894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:36.517894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:36.517894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:36.517894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:36.517894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:36.517894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:36.517894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:36.517894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:00:36.517894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:36.525247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 443/892] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jul 20 09:00:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:36.526339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1879f0-a304-4cdb-89ba-a279f7938371 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:36.577692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc225b6-436c-41ad-8e9a-1304fe319e39 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:36.611997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2edc6-c7bd-4ffd-8ffd-32b1070d4b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:36.612427 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2edc6-c7bd-4ffd-8ffd-32b1070d4b0d None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:36.612708 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2edc6-c7bd-4ffd-8ffd-32b1070d4b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:36.612984 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2edc6-c7bd-4ffd-8ffd-32b1070d4b0d None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:36.613690 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee2edc6-c7bd-4ffd-8ffd-32b1070d4b0d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:36.613998 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee2edc6-c7bd-4ffd-8ffd-32b1070d4b0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:36.615465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee2edc6-c7bd-4ffd-8ffd-32b1070d4b0d None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:36.625521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 450/893] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 09:00:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:36.627457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2edc6-c7bd-4ffd-8ffd-32b1070d4b0d None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:36.658378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1879f0-a304-4cdb-89ba-a279f7938371 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:36.687819 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:36.701133 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380d9ef-92b0-4b0a-996a-2f7d4257fa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:36.701811 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380d9ef-92b0-4b0a-996a-2f7d4257fa9a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:36.702091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380d9ef-92b0-4b0a-996a-2f7d4257fa9a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:36.702365 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380d9ef-92b0-4b0a-996a-2f7d4257fa9a None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:36.703693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380d9ef-92b0-4b0a-996a-2f7d4257fa9a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:36.704011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380d9ef-92b0-4b0a-996a-2f7d4257fa9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:36.705459 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380d9ef-92b0-4b0a-996a-2f7d4257fa9a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:36.729990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 444/894] 10.222.0.44 () {56 vars in 1149 bytes} [Wed Jul 20 09:00:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:36.735273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380d9ef-92b0-4b0a-996a-2f7d4257fa9a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:36.767885 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2edc6-c7bd-4ffd-8ffd-32b1070d4b0d None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:36.813186 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beccb221-fde2-4b98-9eb6-b69368ff7492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:36.813854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccb221-fde2-4b98-9eb6-b69368ff7492 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:36.814135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccb221-fde2-4b98-9eb6-b69368ff7492 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:36.814423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccb221-fde2-4b98-9eb6-b69368ff7492 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:36.815722 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beccb221-fde2-4b98-9eb6-b69368ff7492 None admin] RBAC: Adding query filter params (interface=public, service_id=c8486d513a244eaf965b50ae41909628, region_id=RegionOne) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:36.816055 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beccb221-fde2-4b98-9eb6-b69368ff7492 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:36.817714 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beccb221-fde2-4b98-9eb6-b69368ff7492 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:36.818398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:36.826756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 451/895] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Jul 20 09:00:36 2022] GET /identity/v3/endpoints?service_id=c8486d513a244eaf965b50ae41909628&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:36.830901 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beccb221-fde2-4b98-9eb6-b69368ff7492 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:36.871567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380d9ef-92b0-4b0a-996a-2f7d4257fa9a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:36.908847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2ba11-ff59-413c-82d2-a20df1d3c6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:36.909821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2ba11-ff59-413c-82d2-a20df1d3c6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:36.910843 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2ba11-ff59-413c-82d2-a20df1d3c6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:36.910843 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2ba11-ff59-413c-82d2-a20df1d3c6d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:36.945469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beccb221-fde2-4b98-9eb6-b69368ff7492 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:36.953382 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:36.983449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2ba11-ff59-413c-82d2-a20df1d3c6d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:36.984908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2ba11-ff59-413c-82d2-a20df1d3c6d1 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:36.985947 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3f93e-e2ea-46ab-878f-2d388499c446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:36.986960 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3f93e-e2ea-46ab-878f-2d388499c446 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:36.987564 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3f93e-e2ea-46ab-878f-2d388499c446 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:36.988046 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3f93e-e2ea-46ab-878f-2d388499c446 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:36.988935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3f93e-e2ea-46ab-878f-2d388499c446 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:36.990483 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3f93e-e2ea-46ab-878f-2d388499c446 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:37.003152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 452/896] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jul 20 09:00:36 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:00:37.051257 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:37.100923 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 445/897] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:00:36 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 09:00:37.108118 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3f93e-e2ea-46ab-878f-2d388499c446 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:37.222453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3f93e-e2ea-46ab-878f-2d388499c446 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:37.237120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a831c3-808e-43b1-986e-328e50223bfa None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:37.237422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a831c3-808e-43b1-986e-328e50223bfa None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:37.237682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a831c3-808e-43b1-986e-328e50223bfa None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:37.239349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 446/898] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:37.250005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb99931-e1f2-45d1-a215-d8d514bf38a7 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:37.250603 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb99931-e1f2-45d1-a215-d8d514bf38a7 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:37.251123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb99931-e1f2-45d1-a215-d8d514bf38a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:37.268626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7c2a0-b6fa-46a0-b602-3d31f0fb2caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:37.269436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7c2a0-b6fa-46a0-b602-3d31f0fb2caa None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:37.269728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7c2a0-b6fa-46a0-b602-3d31f0fb2caa None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:37.270012 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7c2a0-b6fa-46a0-b602-3d31f0fb2caa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:37.270697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-acb99931-e1f2-45d1-a215-d8d514bf38a7 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:37.272537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-acb99931-e1f2-45d1-a215-d8d514bf38a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:37.283789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7c2a0-b6fa-46a0-b602-3d31f0fb2caa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:37.285340 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7c2a0-b6fa-46a0-b602-3d31f0fb2caa None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:37.308963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-aab7c2a0-b6fa-46a0-b602-3d31f0fb2caa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:37.308963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:37.308963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:37.308963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:37.308963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:37.308963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:37.308963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:37.308963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:37.308963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:37.308963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:37.308963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:37.308963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:37.308963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:37.308963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:37.308963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:37.310489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:37.312004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:37.312004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:37.312004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:37.312004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:37.312004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 453/899] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:37.316018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7c2a0-b6fa-46a0-b602-3d31f0fb2caa None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:37.345556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:37.364712 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 447/900] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:00:37.393212 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb99931-e1f2-45d1-a215-d8d514bf38a7 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:37.433623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7c2a0-b6fa-46a0-b602-3d31f0fb2caa None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:37.474274 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a49ca-d931-442f-96c2-0ed8fc3e93df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:37.474897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a49ca-d931-442f-96c2-0ed8fc3e93df None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:37.475195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a49ca-d931-442f-96c2-0ed8fc3e93df None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:37.475466 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a49ca-d931-442f-96c2-0ed8fc3e93df None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:37.476207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a49ca-d931-442f-96c2-0ed8fc3e93df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:37.476491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a49ca-d931-442f-96c2-0ed8fc3e93df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:37.477862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a49ca-d931-442f-96c2-0ed8fc3e93df None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:37.494715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 454/901] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:37.500313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a49ca-d931-442f-96c2-0ed8fc3e93df None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:37.516770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:37.531443 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb99931-e1f2-45d1-a215-d8d514bf38a7 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:37.569208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d67f2-cfd2-4c85-bf55-d2a57c66e326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:37.570411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d67f2-cfd2-4c85-bf55-d2a57c66e326 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:37.570840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d67f2-cfd2-4c85-bf55-d2a57c66e326 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:37.571223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d67f2-cfd2-4c85-bf55-d2a57c66e326 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:37.572101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d67f2-cfd2-4c85-bf55-d2a57c66e326 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:37.573591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d67f2-cfd2-4c85-bf55-d2a57c66e326 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:37.587215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-340d67f2-cfd2-4c85-bf55-d2a57c66e326 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:00:37.587215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:37.587215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:37.587215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:37.587215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:37.587215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:37.587215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:37.587215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:37.587215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:00:37.587215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:37.587215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:37.587215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:37.587215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:37.587215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:37.587215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:37.587215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:37.588403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:37.588403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:37.588403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:37.588403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:37.588403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:37.588403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:37.588403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:37.588403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:37.588403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:37.588403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:37.588403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:37.588403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:37.588403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:37.588403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:37.588403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:37.588403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:37.589435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:37.589435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:37.589435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:37.589435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:37.589435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:37.589435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:37.589435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:37.589435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:37.589435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:37.589435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:37.589435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:37.589435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:37.589435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:37.589435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:00:37.589435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:37.597038 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 448/902] 10.222.0.44 () {56 vars in 1132 bytes} [Wed Jul 20 09:00:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:37.598405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d67f2-cfd2-4c85-bf55-d2a57c66e326 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:37.614525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:37.636896 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a49ca-d931-442f-96c2-0ed8fc3e93df None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:37.684212 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046e01b-8a00-42c9-acbf-6307657803e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:37.685270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046e01b-8a00-42c9-acbf-6307657803e0 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:37.685736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046e01b-8a00-42c9-acbf-6307657803e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:37.686815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046e01b-8a00-42c9-acbf-6307657803e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:37.700431 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046e01b-8a00-42c9-acbf-6307657803e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:37.702448 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046e01b-8a00-42c9-acbf-6307657803e0 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:37.723364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-1046e01b-8a00-42c9-acbf-6307657803e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:37.723364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:37.723364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:37.723364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:37.723364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:37.723364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:37.723364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:37.723364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:37.723364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:37.723364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:37.723364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:37.723364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:37.723364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:37.723364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:37.723364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:37.724559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:37.725656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:37.725656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:37.725656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:37.725656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:37.725656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:37.725656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:37.725656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:37.725656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:37.726181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d67f2-cfd2-4c85-bf55-d2a57c66e326 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:37.734154 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 455/903] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:37.735086 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046e01b-8a00-42c9-acbf-6307657803e0 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:37.777136 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56d8cb-cd81-4974-be86-3e39f7fdb107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:37.778964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56d8cb-cd81-4974-be86-3e39f7fdb107 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:37.779488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56d8cb-cd81-4974-be86-3e39f7fdb107 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:37.779969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56d8cb-cd81-4974-be86-3e39f7fdb107 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:37.780906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56d8cb-cd81-4974-be86-3e39f7fdb107 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:37.781990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56d8cb-cd81-4974-be86-3e39f7fdb107 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:37.784044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56d8cb-cd81-4974-be86-3e39f7fdb107 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:37.791593 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 449/904] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jul 20 09:00:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:37.797363 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56d8cb-cd81-4974-be86-3e39f7fdb107 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:37.817182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:37.867481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046e01b-8a00-42c9-acbf-6307657803e0 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:37.915433 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c1745-dc25-498d-a0e3-8454256d43fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:37.917075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c1745-dc25-498d-a0e3-8454256d43fa None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:37.917585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c1745-dc25-498d-a0e3-8454256d43fa None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:37.918070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c1745-dc25-498d-a0e3-8454256d43fa None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:37.918806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56d8cb-cd81-4974-be86-3e39f7fdb107 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:37.919706 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c1745-dc25-498d-a0e3-8454256d43fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:37.920218 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c1745-dc25-498d-a0e3-8454256d43fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:37.922995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c1745-dc25-498d-a0e3-8454256d43fa None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:37.939814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 456/905] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:37.946193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c1745-dc25-498d-a0e3-8454256d43fa None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:37.953353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:37.969288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729df62-0f59-4471-9d44-93de1c527aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:37.970392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729df62-0f59-4471-9d44-93de1c527aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:37.970986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729df62-0f59-4471-9d44-93de1c527aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:37.971598 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729df62-0f59-4471-9d44-93de1c527aa4 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:37.972519 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729df62-0f59-4471-9d44-93de1c527aa4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:37.973051 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729df62-0f59-4471-9d44-93de1c527aa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:37.974889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729df62-0f59-4471-9d44-93de1c527aa4 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:37.986782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 450/906] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jul 20 09:00:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:37.989256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729df62-0f59-4471-9d44-93de1c527aa4 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:38.076768 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c1745-dc25-498d-a0e3-8454256d43fa None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:38.097036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:38.118001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f9b86-8ab9-4ae0-9a5a-6a3e277c47a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:38.118657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f9b86-8ab9-4ae0-9a5a-6a3e277c47a4 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:38.118967 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f9b86-8ab9-4ae0-9a5a-6a3e277c47a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:38.119242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f9b86-8ab9-4ae0-9a5a-6a3e277c47a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:38.120086 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f9b86-8ab9-4ae0-9a5a-6a3e277c47a4 None admin] RBAC: Adding query filter params (role.id=a2636fed60634c3997e3c9213a818395, scope.project.id=a498284274454d25b1b118a88996e522, user.id=6b8db4a7ee284b1aa96d4517dd381e71) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:38.120399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f9b86-8ab9-4ae0-9a5a-6a3e277c47a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:38.121871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f9b86-8ab9-4ae0-9a5a-6a3e277c47a4 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:38.127563 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729df62-0f59-4471-9d44-93de1c527aa4 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:38.135013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 457/907] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:00:37 2022] GET /identity/v3/role_assignments?user.id=6b8db4a7ee284b1aa96d4517dd381e71&scope.project.id=a498284274454d25b1b118a88996e522&role.id=a2636fed60634c3997e3c9213a818395 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:38.171871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994224a-ae20-4d22-9362-7513efcdd275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:38.172546 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994224a-ae20-4d22-9362-7513efcdd275 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:38.172831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994224a-ae20-4d22-9362-7513efcdd275 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:38.173108 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994224a-ae20-4d22-9362-7513efcdd275 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:38.173349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb07cb-8006-4e3d-a62f-a8393d7a1232 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:38.173638 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb07cb-8006-4e3d-a62f-a8393d7a1232 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:38.173913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb07cb-8006-4e3d-a62f-a8393d7a1232 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:38.174025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994224a-ae20-4d22-9362-7513efcdd275 None admin] RBAC: Adding query filter params (interface=public, service_id=491a05ee21d4451f86916c608f075169, region_id=RegionOne) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:38.174269 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994224a-ae20-4d22-9362-7513efcdd275 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:38.175563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 458/908] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:38.175765 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994224a-ae20-4d22-9362-7513efcdd275 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:38.182705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0319a5-0250-491a-a7e1-18fe83b87c25 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:38.182705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0319a5-0250-491a-a7e1-18fe83b87c25 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:38.182705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0319a5-0250-491a-a7e1-18fe83b87c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:38.190065 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 451/909] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Jul 20 09:00:37 2022] GET /identity/v3/endpoints?service_id=491a05ee21d4451f86916c608f075169&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:38.192724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994224a-ae20-4d22-9362-7513efcdd275 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:38.207496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-8d0319a5-0250-491a-a7e1-18fe83b87c25 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:38.208742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8d0319a5-0250-491a-a7e1-18fe83b87c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:38.281531 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:38.292964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 459/910] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:38.315481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0319a5-0250-491a-a7e1-18fe83b87c25 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:38.334588 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994224a-ae20-4d22-9362-7513efcdd275 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:38.382340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14354837-dcb1-46fc-a4c0-af5f90aa80eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:38.384003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14354837-dcb1-46fc-a4c0-af5f90aa80eb None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:38.384516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14354837-dcb1-46fc-a4c0-af5f90aa80eb None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:38.385011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14354837-dcb1-46fc-a4c0-af5f90aa80eb None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:38.385890 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14354837-dcb1-46fc-a4c0-af5f90aa80eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:38.388840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14354837-dcb1-46fc-a4c0-af5f90aa80eb None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:38.397161 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 452/911] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jul 20 09:00:38 2022] GET /identity/v3/services => generated 1268 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:00:38.434737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0319a5-0250-491a-a7e1-18fe83b87c25 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:38.475021 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c67a47-6723-411e-ba82-4a1632354903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:38.475711 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c67a47-6723-411e-ba82-4a1632354903 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:38.475995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c67a47-6723-411e-ba82-4a1632354903 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:38.476275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c67a47-6723-411e-ba82-4a1632354903 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:38.477026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c67a47-6723-411e-ba82-4a1632354903 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:38.478428 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c67a47-6723-411e-ba82-4a1632354903 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:38.495576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 460/912] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jul 20 09:00:38 2022] POST /identity/v3/services => generated 241 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:00:39.126420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57658485-64ba-42ae-8284-c8a4b70b08d4 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:39.126960 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57658485-64ba-42ae-8284-c8a4b70b08d4 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:39.127067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57658485-64ba-42ae-8284-c8a4b70b08d4 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:39.128679 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 453/913] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:39.133880 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b380470-d281-4c9e-ab39-405e5eb4edbf None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:39.134315 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b380470-d281-4c9e-ab39-405e5eb4edbf None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:39.134487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b380470-d281-4c9e-ab39-405e5eb4edbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:39.152260 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-9b380470-d281-4c9e-ab39-405e5eb4edbf None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:39.153696 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-9b380470-d281-4c9e-ab39-405e5eb4edbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:39.208047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:39.232355 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 461/914] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:39.255734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57658485-64ba-42ae-8284-c8a4b70b08d4 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:39.269461 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:39.373649 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57658485-64ba-42ae-8284-c8a4b70b08d4 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:39.413271 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677eb19a-a2f8-4ba1-99d6-50963ccbd476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:39.414398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677eb19a-a2f8-4ba1-99d6-50963ccbd476 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:39.414912 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677eb19a-a2f8-4ba1-99d6-50963ccbd476 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:39.415387 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677eb19a-a2f8-4ba1-99d6-50963ccbd476 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:39.416256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677eb19a-a2f8-4ba1-99d6-50963ccbd476 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:39.417798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677eb19a-a2f8-4ba1-99d6-50963ccbd476 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-677eb19a-a2f8-4ba1-99d6-50963ccbd476 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:39.430221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:39.431230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:39.431230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:39.431230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:39.431230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:39.431230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:39.431230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:39.431230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:39.431230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:39.431230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:39.431230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:39.431230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:39.431230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:39.431230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:39.431230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:39.431230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:39.431230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:39.432047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:39.432047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:39.432047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:39.432047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:39.432047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:39.432047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:39.432047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:39.432047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:39.432047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:39.432047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:39.432047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:39.432047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:39.432047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 09:00:39.432047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:39.434930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 454/915] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 09:00:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:39.442858 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b380470-d281-4c9e-ab39-405e5eb4edbf None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:39.544988 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b380470-d281-4c9e-ab39-405e5eb4edbf None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:39.582996 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b00a6-cf25-4a2d-8c3c-d706916d2e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:39.583666 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b00a6-cf25-4a2d-8c3c-d706916d2e22 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:39.583949 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b00a6-cf25-4a2d-8c3c-d706916d2e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:39.584225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b00a6-cf25-4a2d-8c3c-d706916d2e22 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:39.584941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b00a6-cf25-4a2d-8c3c-d706916d2e22 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:39.585256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b00a6-cf25-4a2d-8c3c-d706916d2e22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:39.586628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b00a6-cf25-4a2d-8c3c-d706916d2e22 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:39.596580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 462/916] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 09:00:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:39.597941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677eb19a-a2f8-4ba1-99d6-50963ccbd476 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:39.675139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:39.708243 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677eb19a-a2f8-4ba1-99d6-50963ccbd476 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:39.742589 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7181b1-e471-4f95-bc6c-a7c3d9add854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:39.743600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7181b1-e471-4f95-bc6c-a7c3d9add854 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:39.744073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7181b1-e471-4f95-bc6c-a7c3d9add854 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:39.744536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7181b1-e471-4f95-bc6c-a7c3d9add854 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:39.745421 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7181b1-e471-4f95-bc6c-a7c3d9add854 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:39.745935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7181b1-e471-4f95-bc6c-a7c3d9add854 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:39.748559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7181b1-e471-4f95-bc6c-a7c3d9add854 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:39.758469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 455/917] 10.222.0.44 () {56 vars in 1114 bytes} [Wed Jul 20 09:00:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:39.760063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b00a6-cf25-4a2d-8c3c-d706916d2e22 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:39.807558 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:39.862086 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b00a6-cf25-4a2d-8c3c-d706916d2e22 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:39.897136 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127aaa4-3076-416b-a0ed-cabf49d1018c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:39.897796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127aaa4-3076-416b-a0ed-cabf49d1018c None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:39.898079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127aaa4-3076-416b-a0ed-cabf49d1018c None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:39.898358 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127aaa4-3076-416b-a0ed-cabf49d1018c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:39.899135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127aaa4-3076-416b-a0ed-cabf49d1018c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:39.900508 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127aaa4-3076-416b-a0ed-cabf49d1018c None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:39.924023 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 463/918] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jul 20 09:00:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:00:40.230477 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2dde2-2904-4a9a-a139-1e009338e7cb None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:40.231681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2dde2-2904-4a9a-a139-1e009338e7cb None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:40.232164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2dde2-2904-4a9a-a139-1e009338e7cb None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:40.233966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 456/919] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:40.239658 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341cb84-1e54-4b86-be68-074b07daa4d1 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:40.240496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341cb84-1e54-4b86-be68-074b07daa4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:40.240991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341cb84-1e54-4b86-be68-074b07daa4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:40.258987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-f341cb84-1e54-4b86-be68-074b07daa4d1 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:40.260504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f341cb84-1e54-4b86-be68-074b07daa4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:40.342777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 464/920] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:40.361618 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2dde2-2904-4a9a-a139-1e009338e7cb None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:40.470856 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2dde2-2904-4a9a-a139-1e009338e7cb None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:40.506380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425a0ff-e91d-479b-8260-29249ec1bca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:40.507624 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425a0ff-e91d-479b-8260-29249ec1bca9 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:40.508120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425a0ff-e91d-479b-8260-29249ec1bca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:40.508600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425a0ff-e91d-479b-8260-29249ec1bca9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:40.511532 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:40.521101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9425a0ff-e91d-479b-8260-29249ec1bca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:40.530326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-9425a0ff-e91d-479b-8260-29249ec1bca9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:00:40.530326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:40.530326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:40.530326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:40.530326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:40.530326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:40.530326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:40.530326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:40.530326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:40.530326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:40.530326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:40.530326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:40.530326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:40.530326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:40.530326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:40.530326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:40.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:40.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:40.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:40.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:40.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:40.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:40.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:40.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:40.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:40.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:40.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:40.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:40.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:40.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:40.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:40.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:40.532293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:40.532293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:40.532293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:40.532293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:40.532293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:40.532293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:40.532293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:40.532293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:40.532293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:40.532293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:40.532293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:00:40.532293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:40.536439 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 457/921] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Jul 20 09:00:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:40.537827 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341cb84-1e54-4b86-be68-074b07daa4d1 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:40.555756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e69207-1a81-4352-8e11-c68cb932985a None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:40.556044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e69207-1a81-4352-8e11-c68cb932985a None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:40.556323 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e69207-1a81-4352-8e11-c68cb932985a None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:40.558240 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 458/922] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:40.564151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48204733-8ad2-4f42-98fd-06a2b3727b3d None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:40.564695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48204733-8ad2-4f42-98fd-06a2b3727b3d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:40.565200 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48204733-8ad2-4f42-98fd-06a2b3727b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:40.575631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:40.584483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-48204733-8ad2-4f42-98fd-06a2b3727b3d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:40.585988 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-48204733-8ad2-4f42-98fd-06a2b3727b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:40.658369 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341cb84-1e54-4b86-be68-074b07daa4d1 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:40.660690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 459/923] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:00:40.661955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f3d6d-d72e-4670-b260-2bcdcd7e2254 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:40.662246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f3d6d-d72e-4670-b260-2bcdcd7e2254 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:40.662520 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f3d6d-d72e-4670-b260-2bcdcd7e2254 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:40.663925 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 460/924] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:40.670385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40749d80-d2d7-4dfe-bdaa-1f8e77ba47bc None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:40.670872 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40749d80-d2d7-4dfe-bdaa-1f8e77ba47bc None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:40.671161 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40749d80-d2d7-4dfe-bdaa-1f8e77ba47bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:40.688071 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-40749d80-d2d7-4dfe-bdaa-1f8e77ba47bc None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:40.689248 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-40749d80-d2d7-4dfe-bdaa-1f8e77ba47bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:40.709204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b390c-a7aa-454b-aa8d-d2ad771d1449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:40.710612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b390c-a7aa-454b-aa8d-d2ad771d1449 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:40.711345 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b390c-a7aa-454b-aa8d-d2ad771d1449 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:40.711796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b390c-a7aa-454b-aa8d-d2ad771d1449 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:40.712885 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b390c-a7aa-454b-aa8d-d2ad771d1449 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:40.713685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b390c-a7aa-454b-aa8d-d2ad771d1449 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:40.716198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b390c-a7aa-454b-aa8d-d2ad771d1449 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:40.726716 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 465/925] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Jul 20 09:00:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:40.727686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b390c-a7aa-454b-aa8d-d2ad771d1449 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:40.767112 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 461/926] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:00:40.769392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40749d80-d2d7-4dfe-bdaa-1f8e77ba47bc None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:40.828756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:40.833711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:40.841598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b390c-a7aa-454b-aa8d-d2ad771d1449 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:40.877651 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40749d80-d2d7-4dfe-bdaa-1f8e77ba47bc None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:40.878750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eeb818-9161-4223-9b8c-8acd6f002548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:40.879771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eeb818-9161-4223-9b8c-8acd6f002548 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:40.880075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eeb818-9161-4223-9b8c-8acd6f002548 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:40.880361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eeb818-9161-4223-9b8c-8acd6f002548 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:40.881017 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eeb818-9161-4223-9b8c-8acd6f002548 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:40.882396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eeb818-9161-4223-9b8c-8acd6f002548 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:40.896146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-34eeb818-9161-4223-9b8c-8acd6f002548 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:00:40.896146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:40.896146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:40.896146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:40.896146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:40.896146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:40.896146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:40.896146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:40.896146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:00:40.896146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:40.896146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:40.896146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:40.896146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:40.896146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:40.896146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:40.896146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:40.897148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:40.897148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:40.897148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:40.897148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:40.897148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:40.897148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:40.897148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:40.897148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:40.897148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:40.897148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:40.897148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:40.897148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:40.897148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:40.897148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:40.897148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:40.897148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:40.903681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:40.903681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:40.903681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:40.903681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:40.903681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:40.903681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:40.903681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:40.903681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:40.903681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:40.903681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:40.903681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:40.903681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:40.903681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:40.903681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 09:00:40.903681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:40.903681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 466/927] 10.222.0.44 () {56 vars in 1132 bytes} [Wed Jul 20 09:00:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:40.904548 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eeb818-9161-4223-9b8c-8acd6f002548 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:40.920552 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc658559-8fc5-4337-9540-06a815b03f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:40.921432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc658559-8fc5-4337-9540-06a815b03f6a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:40.921791 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc658559-8fc5-4337-9540-06a815b03f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:40.922144 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc658559-8fc5-4337-9540-06a815b03f6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:40.934153 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc658559-8fc5-4337-9540-06a815b03f6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:40.935616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc658559-8fc5-4337-9540-06a815b03f6a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:40.954067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-fc658559-8fc5-4337-9540-06a815b03f6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:40.954067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:40.954067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:40.954067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:40.954067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:40.954067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:40.954067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:40.954067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:40.954067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:40.954067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:40.954067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:40.954067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:40.954067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:40.954067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:40.954067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:40.955050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:40.955946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:40.955946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:40.955946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:40.955946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:40.955946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 462/928] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:40.956255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc658559-8fc5-4337-9540-06a815b03f6a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:41.023331 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eeb818-9161-4223-9b8c-8acd6f002548 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:41.060799 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e4279-835f-4190-a190-f8aa07059323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:41.061636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e4279-835f-4190-a190-f8aa07059323 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:41.061927 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e4279-835f-4190-a190-f8aa07059323 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:41.062209 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e4279-835f-4190-a190-f8aa07059323 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:41.062917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e4279-835f-4190-a190-f8aa07059323 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:41.063088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc658559-8fc5-4337-9540-06a815b03f6a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:41.064350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e4279-835f-4190-a190-f8aa07059323 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:41.073222 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-b80e4279-835f-4190-a190-f8aa07059323 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:41.079860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:41.081035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:41.081035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:41.081035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:41.081035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:41.081035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:41.081035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:41.081035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:41.081035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:41.081035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:41.081035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:41.081035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:41.081035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:41.081035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:41.081035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:41.081035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:41.081035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:41.081908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:41.081908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:41.081908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:41.081908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:41.081908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:41.081908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:41.081908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:41.081908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:41.081908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:41.081908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:41.081908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:41.081908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:41.081908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:00:41.081908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:41.081908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 467/929] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jul 20 09:00:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:41.082751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e4279-835f-4190-a190-f8aa07059323 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:41.104749 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01ee2e-dfca-4d7b-91ba-6271603bb232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:41.105856 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01ee2e-dfca-4d7b-91ba-6271603bb232 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:41.107001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:41.107205 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01ee2e-dfca-4d7b-91ba-6271603bb232 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:41.107447 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01ee2e-dfca-4d7b-91ba-6271603bb232 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:41.108165 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01ee2e-dfca-4d7b-91ba-6271603bb232 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:41.108475 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01ee2e-dfca-4d7b-91ba-6271603bb232 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:41.110235 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01ee2e-dfca-4d7b-91ba-6271603bb232 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:41.121143 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 463/930] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jul 20 09:00:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:41.121143 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01ee2e-dfca-4d7b-91ba-6271603bb232 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:41.196110 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e4279-835f-4190-a190-f8aa07059323 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:41.231829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b23fe8-0c52-48e8-95de-87788a07d030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:41.232498 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b23fe8-0c52-48e8-95de-87788a07d030 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:41.232784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b23fe8-0c52-48e8-95de-87788a07d030 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:41.233067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b23fe8-0c52-48e8-95de-87788a07d030 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:41.233787 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b23fe8-0c52-48e8-95de-87788a07d030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:41.234097 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b23fe8-0c52-48e8-95de-87788a07d030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:41.234982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01ee2e-dfca-4d7b-91ba-6271603bb232 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:41.235716 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b23fe8-0c52-48e8-95de-87788a07d030 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:41.248242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 468/931] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:41.249049 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b23fe8-0c52-48e8-95de-87788a07d030 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:41.281993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24611ba7-1b0c-41c3-ac8c-31c42c13e3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:41.282585 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24611ba7-1b0c-41c3-ac8c-31c42c13e3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:41.282900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24611ba7-1b0c-41c3-ac8c-31c42c13e3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:41.283186 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24611ba7-1b0c-41c3-ac8c-31c42c13e3d1 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:41.283897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24611ba7-1b0c-41c3-ac8c-31c42c13e3d1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:41.284204 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24611ba7-1b0c-41c3-ac8c-31c42c13e3d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:41.285555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24611ba7-1b0c-41c3-ac8c-31c42c13e3d1 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:41.290192 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 464/932] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jul 20 09:00:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:41.290993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24611ba7-1b0c-41c3-ac8c-31c42c13e3d1 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:41.331310 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:41.340761 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:41.364667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b23fe8-0c52-48e8-95de-87788a07d030 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:41.405608 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d6bff-bb07-4bb6-934e-750f7d5093e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:41.406148 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24611ba7-1b0c-41c3-ac8c-31c42c13e3d1 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:41.406267 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d6bff-bb07-4bb6-934e-750f7d5093e7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:41.406550 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d6bff-bb07-4bb6-934e-750f7d5093e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:41.406891 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d6bff-bb07-4bb6-934e-750f7d5093e7 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:41.407648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d6bff-bb07-4bb6-934e-750f7d5093e7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:41.407910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d6bff-bb07-4bb6-934e-750f7d5093e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:41.409394 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d6bff-bb07-4bb6-934e-750f7d5093e7 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:41.414720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 469/933] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jul 20 09:00:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:41.415406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d6bff-bb07-4bb6-934e-750f7d5093e7 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:41.447306 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410505ad-4050-42cb-8457-02df095d2ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:41.448122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410505ad-4050-42cb-8457-02df095d2ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:41.448430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410505ad-4050-42cb-8457-02df095d2ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:41.448682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410505ad-4050-42cb-8457-02df095d2ca0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:41.456790 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410505ad-4050-42cb-8457-02df095d2ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:41.458219 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410505ad-4050-42cb-8457-02df095d2ca0 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:41.473805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-410505ad-4050-42cb-8457-02df095d2ca0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:00:41.473805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:41.473805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:41.473805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:41.473805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:41.473805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:41.473805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:41.473805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:41.473805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:41.473805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:41.473805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:41.473805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:41.473805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:41.473805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:41.473805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:41.473805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:41.474816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:41.475955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:41.475955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:41.475955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:41.475955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:41.475955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:41.475955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:41.475955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:41.475955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:41.475955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:00:41.475955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:41.479989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 465/934] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jul 20 09:00:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:41.480711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410505ad-4050-42cb-8457-02df095d2ca0 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:41.529395 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d6bff-bb07-4bb6-934e-750f7d5093e7 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:41.573709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f64dde-cd57-4db5-8985-23457e8b8bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:41.574378 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f64dde-cd57-4db5-8985-23457e8b8bda None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:41.575207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f64dde-cd57-4db5-8985-23457e8b8bda None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:41.575523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f64dde-cd57-4db5-8985-23457e8b8bda None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:41.576557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f64dde-cd57-4db5-8985-23457e8b8bda None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:41.576870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f64dde-cd57-4db5-8985-23457e8b8bda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:41.578244 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f64dde-cd57-4db5-8985-23457e8b8bda None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:41.587939 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 470/935] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jul 20 09:00:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:41.587939 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f64dde-cd57-4db5-8985-23457e8b8bda None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:41.596881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:41.599796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:41.604417 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410505ad-4050-42cb-8457-02df095d2ca0 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:41.640631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017b6e5-9474-4226-be01-e5cba0d0e41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:41.641292 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017b6e5-9474-4226-be01-e5cba0d0e41a None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:41.641576 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017b6e5-9474-4226-be01-e5cba0d0e41a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:41.641851 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017b6e5-9474-4226-be01-e5cba0d0e41a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:41.642589 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017b6e5-9474-4226-be01-e5cba0d0e41a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:41.644025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f017b6e5-9474-4226-be01-e5cba0d0e41a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:41.662496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 466/936] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jul 20 09:00:41 2022] POST /identity/v3/endpoints => generated 347 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:00:41.663370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017b6e5-9474-4226-be01-e5cba0d0e41a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:41.713431 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f64dde-cd57-4db5-8985-23457e8b8bda None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:41.769234 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3f60e-ea68-4103-b5fe-5aebd0a43828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:41.770307 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3f60e-ea68-4103-b5fe-5aebd0a43828 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:41.770939 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3f60e-ea68-4103-b5fe-5aebd0a43828 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:41.771431 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3f60e-ea68-4103-b5fe-5aebd0a43828 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:41.772681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3f60e-ea68-4103-b5fe-5aebd0a43828 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:41.773194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3f60e-ea68-4103-b5fe-5aebd0a43828 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:41.774908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3f60e-ea68-4103-b5fe-5aebd0a43828 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:41.802723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 471/937] 10.222.0.44 () {56 vars in 1149 bytes} [Wed Jul 20 09:00:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:41.804346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017b6e5-9474-4226-be01-e5cba0d0e41a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:41.808296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3f60e-ea68-4103-b5fe-5aebd0a43828 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:41.846409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c214342-fcd4-487b-b770-79574570fa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:41.847288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c214342-fcd4-487b-b770-79574570fa01 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:41.847577 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c214342-fcd4-487b-b770-79574570fa01 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:41.847859 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c214342-fcd4-487b-b770-79574570fa01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:41.849042 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c214342-fcd4-487b-b770-79574570fa01 None admin] RBAC: Adding query filter params (interface=public, service_id=7d362264b4b84421af8604a13b3c6722, region_id=RegionOne) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:41.849362 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c214342-fcd4-487b-b770-79574570fa01 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:41.850759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c214342-fcd4-487b-b770-79574570fa01 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:41.857270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 467/938] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Jul 20 09:00:41 2022] GET /identity/v3/endpoints?service_id=7d362264b4b84421af8604a13b3c6722&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:41.864143 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c214342-fcd4-487b-b770-79574570fa01 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:41.889957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:41.897887 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:41.953926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3f60e-ea68-4103-b5fe-5aebd0a43828 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:41.982566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c214342-fcd4-487b-b770-79574570fa01 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:41.999155 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5a640-a640-4d0c-9397-28b5a6b19920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:41.999975 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5a640-a640-4d0c-9397-28b5a6b19920 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:42.000403 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5a640-a640-4d0c-9397-28b5a6b19920 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:42.000691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5a640-a640-4d0c-9397-28b5a6b19920 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:42.014225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5a640-a640-4d0c-9397-28b5a6b19920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:42.015871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5a640-a640-4d0c-9397-28b5a6b19920 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:42.018545 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104c8e2-53da-4690-a0cd-4df79a81a6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:42.019250 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104c8e2-53da-4690-a0cd-4df79a81a6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:42.019532 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104c8e2-53da-4690-a0cd-4df79a81a6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:42.019826 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104c8e2-53da-4690-a0cd-4df79a81a6b3 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:42.020513 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104c8e2-53da-4690-a0cd-4df79a81a6b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:42.021843 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104c8e2-53da-4690-a0cd-4df79a81a6b3 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:42.028663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 468/939] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jul 20 09:00:41 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:00:42.029710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790c254-7deb-4290-80c0-cb4bb34cd12d None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:42.029998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790c254-7deb-4290-80c0-cb4bb34cd12d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:42.030276 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790c254-7deb-4290-80c0-cb4bb34cd12d None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:42.031702 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 469/940] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:42.038723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410b250-8e7a-49d9-922e-81723911d867 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:42.038723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410b250-8e7a-49d9-922e-81723911d867 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:42.038723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410b250-8e7a-49d9-922e-81723911d867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:42.039056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-1bd5a640-a640-4d0c-9397-28b5a6b19920 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:42.039056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:42.039056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:42.039056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:42.039056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:42.039056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:42.039056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:42.039056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:42.039056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:42.039056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:42.039056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:42.039056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:42.039056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:42.039056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:42.039056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:42.039873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:42.040789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:42.040789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:42.040789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:42.040789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:42.040789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 472/941] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:42.047062 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5a640-a640-4d0c-9397-28b5a6b19920 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:42.063157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-f410b250-8e7a-49d9-922e-81723911d867 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:42.064387 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f410b250-8e7a-49d9-922e-81723911d867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:42.166825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 470/942] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:42.173310 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5a640-a640-4d0c-9397-28b5a6b19920 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:42.189394 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:42.195162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410b250-8e7a-49d9-922e-81723911d867 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:42.209110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:42.226923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fff52-3739-4927-8bb9-858bb62e3e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:42.227588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fff52-3739-4927-8bb9-858bb62e3e26 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:42.227871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fff52-3739-4927-8bb9-858bb62e3e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:42.228150 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fff52-3739-4927-8bb9-858bb62e3e26 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:42.229375 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fff52-3739-4927-8bb9-858bb62e3e26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:42.229694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fff52-3739-4927-8bb9-858bb62e3e26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:42.231243 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fff52-3739-4927-8bb9-858bb62e3e26 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:42.244163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 473/943] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:42.250078 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fff52-3739-4927-8bb9-858bb62e3e26 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:42.335153 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410b250-8e7a-49d9-922e-81723911d867 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:42.367491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fff52-3739-4927-8bb9-858bb62e3e26 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:42.380230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c9f67-0fac-4bf1-9842-712594fbadac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:42.380942 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c9f67-0fac-4bf1-9842-712594fbadac None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:42.381228 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c9f67-0fac-4bf1-9842-712594fbadac None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:42.381509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c9f67-0fac-4bf1-9842-712594fbadac None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:42.382234 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c9f67-0fac-4bf1-9842-712594fbadac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:42.384500 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c9f67-0fac-4bf1-9842-712594fbadac None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:42.386184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: WARNING keystone.server.flask.common [None req-d63c9f67-0fac-4bf1-9842-712594fbadac None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 09:00:42.411796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.sql.core [None req-d63c9f67-0fac-4bf1-9842-712594fbadac None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 09:00:42.411796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 09:00:42.411796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [parameters: {'id': 'bd37584024b747169b5059a8dccbe203', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 09:00:42.411796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80333) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 09:00:42.412336 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f941936-d3b3-411f-a36c-468d5efb2ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:42.413150 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f941936-d3b3-411f-a36c-468d5efb2ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:42.413434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f941936-d3b3-411f-a36c-468d5efb2ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:42.413713 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f941936-d3b3-411f-a36c-468d5efb2ac8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:42.421210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f941936-d3b3-411f-a36c-468d5efb2ac8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:42.422719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f941936-d3b3-411f-a36c-468d5efb2ac8 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:42.433613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-d63c9f67-0fac-4bf1-9842-712594fbadac None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bd37584024b747169b5059a8dccbe203.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bd37584024b747169b5059a8dccbe203. Jul 20 09:00:42.433613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:42.433613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 09:00:42.433613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 09:00:42.433613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 09:00:42.433613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 09:00:42.433613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 09:00:42.433613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 09:00:42.433613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 09:00:42.433613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application conn.query(q) Jul 20 09:00:42.433613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 09:00:42.433613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 09:00:42.433613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application result.read() Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 20 09:00:42.434734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application next(self.gen) Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application yield resource Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application next(self.gen) Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise self.value Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application session.commit() Jul 20 09:00:42.435874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self.session.flush() Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application compat.raise_( Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 09:00:42.436817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 09:00:42.439063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 09:00:42.439063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 09:00:42.439063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 09:00:42.439063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 09:00:42.439063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 09:00:42.439063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 09:00:42.439063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 09:00:42.439063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 09:00:42.439063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 09:00:42.439063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 09:00:42.439063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 09:00:42.439063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 09:00:42.439063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 09:00:42.439063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 09:00:42.439063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application conn.query(q) Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application result.read() Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 09:00:42.440973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 09:00:42.443113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 09:00:42.443113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 09:00:42.443113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 09:00:42.443113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 09:00:42.443113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 09:00:42.443113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 09:00:42.443113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 09:00:42.443113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 09:00:42.443113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [parameters: {'id': 'bd37584024b747169b5059a8dccbe203', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 09:00:42.443113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 09:00:42.443113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:42.443113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:42.443113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:42.443113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:42.443113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:42.445328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:42.448837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:42.448837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 20 09:00:42.448837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 20 09:00:42.448837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:42.448837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:42.448837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 20 09:00:42.448837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 20 09:00:42.448837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bd37584024b747169b5059a8dccbe203. Jul 20 09:00:42.448837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:42.450499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-5f941936-d3b3-411f-a36c-468d5efb2ac8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:42.450499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:42.450499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:42.450499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:42.450499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:42.450499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:42.450499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:42.450499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:42.450499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:42.450499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:42.450499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:42.450499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:42.450499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:42.450499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:42.450499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:42.458457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:42.460549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:42.460549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:42.460549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:42.460549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:42.460549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:42.460549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:42.460549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:42.460549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:42.460549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 474/944] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:42.460549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f941936-d3b3-411f-a36c-468d5efb2ac8 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:42.461908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 471/945] 10.222.0.44 () {60 vars in 1140 bytes} [Wed Jul 20 09:00:42 2022] POST /identity/v3/projects => generated 262 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:42.461908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c9f67-0fac-4bf1-9842-712594fbadac None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:42.489950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:42.536894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:42.578625 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f941936-d3b3-411f-a36c-468d5efb2ac8 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:42.580593 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c9f67-0fac-4bf1-9842-712594fbadac None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:42.617596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc8713-945b-40b9-b9e3-8919fb96046e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:42.618903 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc8713-945b-40b9-b9e3-8919fb96046e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:42.619493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc8713-945b-40b9-b9e3-8919fb96046e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:42.620052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc8713-945b-40b9-b9e3-8919fb96046e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:42.622981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587303f-0e2d-46db-8ecb-a38ee8452b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:42.623997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587303f-0e2d-46db-8ecb-a38ee8452b02 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:42.624283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587303f-0e2d-46db-8ecb-a38ee8452b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:42.624962 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587303f-0e2d-46db-8ecb-a38ee8452b02 None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:42.625762 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587303f-0e2d-46db-8ecb-a38ee8452b02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:42.626072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587303f-0e2d-46db-8ecb-a38ee8452b02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:42.627538 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587303f-0e2d-46db-8ecb-a38ee8452b02 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:42.630056 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc8713-945b-40b9-b9e3-8919fb96046e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:42.632033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc8713-945b-40b9-b9e3-8919fb96046e None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:42.645773 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 475/946] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:42.646622 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587303f-0e2d-46db-8ecb-a38ee8452b02 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:42.647844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-fcbc8713-945b-40b9-b9e3-8919fb96046e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:42.647844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:42.647844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:42.647844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:42.647844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:42.647844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:42.647844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:42.647844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:42.647844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:42.647844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:42.647844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:42.647844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:42.647844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:42.647844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:42.647844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:42.648688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:42.649578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:42.649578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:42.649578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:42.649578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:42.649578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:42.649578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:42.649578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:42.649578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:42.657118 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 472/947] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:42.658235 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc8713-945b-40b9-b9e3-8919fb96046e None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:42.769483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc8713-945b-40b9-b9e3-8919fb96046e None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:42.773803 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:42.785924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587303f-0e2d-46db-8ecb-a38ee8452b02 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:42.801019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:42.811600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970e7fa-2f83-4029-b29b-68697dd8c1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:42.812668 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970e7fa-2f83-4029-b29b-68697dd8c1fd None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:42.813169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970e7fa-2f83-4029-b29b-68697dd8c1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:42.813669 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970e7fa-2f83-4029-b29b-68697dd8c1fd None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:42.814620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970e7fa-2f83-4029-b29b-68697dd8c1fd None admin] RBAC: Adding query filter params (name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:42.815224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970e7fa-2f83-4029-b29b-68697dd8c1fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:42.816938 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5970e7fa-2f83-4029-b29b-68697dd8c1fd None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:42.827591 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfdc4f-6f0a-4b45-a009-3fe98612577c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:42.828543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfdc4f-6f0a-4b45-a009-3fe98612577c None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:42.828871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfdc4f-6f0a-4b45-a009-3fe98612577c None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:42.829149 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfdc4f-6f0a-4b45-a009-3fe98612577c None admin] PATH_INFO: `/v3/projects/a498284274454d25b1b118a88996e522/users/6b8db4a7ee284b1aa96d4517dd381e71/roles/a2636fed60634c3997e3c9213a818395` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:42.831956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfdc4f-6f0a-4b45-a009-3fe98612577c None admin] RBAC: Authorizing `identity:create_grant(project_id=a498284274454d25b1b118a88996e522, user_id=6b8db4a7ee284b1aa96d4517dd381e71, role_id=a2636fed60634c3997e3c9213a818395)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:42.833453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfdc4f-6f0a-4b45-a009-3fe98612577c None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:42.842086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 473/948] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jul 20 09:00:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:42.850144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 476/949] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 09:00:42 2022] PUT /identity/v3/projects/a498284274454d25b1b118a88996e522/users/6b8db4a7ee284b1aa96d4517dd381e71/roles/a2636fed60634c3997e3c9213a818395 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:00:43.841812 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099fabb-18eb-47ff-b771-fe319edff217 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:43.842857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099fabb-18eb-47ff-b771-fe319edff217 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:43.843370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099fabb-18eb-47ff-b771-fe319edff217 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:43.845218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 474/950] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:43.850555 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d00e9f-c195-4524-9ed0-5bfc3275836a None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:43.851414 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d00e9f-c195-4524-9ed0-5bfc3275836a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:43.851897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d00e9f-c195-4524-9ed0-5bfc3275836a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:43.869178 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-80d00e9f-c195-4524-9ed0-5bfc3275836a None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:43.870718 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-80d00e9f-c195-4524-9ed0-5bfc3275836a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:43.941272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 477/951] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:43.963979 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099fabb-18eb-47ff-b771-fe319edff217 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:44.077251 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099fabb-18eb-47ff-b771-fe319edff217 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:44.112398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c9618-897b-4ddc-9eb5-9a5b35743006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:44.113194 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c9618-897b-4ddc-9eb5-9a5b35743006 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:44.113471 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c9618-897b-4ddc-9eb5-9a5b35743006 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:44.113743 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c9618-897b-4ddc-9eb5-9a5b35743006 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:44.114399 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c9618-897b-4ddc-9eb5-9a5b35743006 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:44.115804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c9618-897b-4ddc-9eb5-9a5b35743006 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-5b1c9618-897b-4ddc-9eb5-9a5b35743006 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:44.127525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:44.128496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:44.128496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:44.128496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:44.128496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:44.128496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:44.128496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:44.128496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:44.128496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:44.128496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:44.128496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:44.128496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:44.128496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:44.128496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:44.128496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:44.128496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:44.128496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:44.129318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:44.129318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:44.129318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:44.129318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:44.129318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:44.129318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:44.129318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:44.129318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:44.129318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:44.129318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:44.129318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:44.129318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:44.129318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:00:44.129318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:44.129318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 475/952] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:44.136716 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d00e9f-c195-4524-9ed0-5bfc3275836a None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:44.201840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100b3db-5a71-4325-b800-e7131e14899c None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:44.202149 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100b3db-5a71-4325-b800-e7131e14899c None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:44.202414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100b3db-5a71-4325-b800-e7131e14899c None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:44.204114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 476/953] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:44.209922 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ab8b5-1713-40ed-a03a-778b71c22d48 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:44.211014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ab8b5-1713-40ed-a03a-778b71c22d48 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:44.211303 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ab8b5-1713-40ed-a03a-778b71c22d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:44.223063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:44.232222 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-350ab8b5-1713-40ed-a03a-778b71c22d48 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:44.234215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-350ab8b5-1713-40ed-a03a-778b71c22d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:44.242070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d00e9f-c195-4524-9ed0-5bfc3275836a None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:44.279514 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0257b3e-a206-4999-b8b2-020f7c45eac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:44.280348 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0257b3e-a206-4999-b8b2-020f7c45eac5 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:44.280633 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0257b3e-a206-4999-b8b2-020f7c45eac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:44.280908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0257b3e-a206-4999-b8b2-020f7c45eac5 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:44.281620 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0257b3e-a206-4999-b8b2-020f7c45eac5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:44.281925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0257b3e-a206-4999-b8b2-020f7c45eac5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:44.283325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0257b3e-a206-4999-b8b2-020f7c45eac5 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:44.288750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 478/954] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jul 20 09:00:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:44.294451 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:44.296328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0257b3e-a206-4999-b8b2-020f7c45eac5 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:44.321247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 477/955] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:44.344163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ab8b5-1713-40ed-a03a-778b71c22d48 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:44.415240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0257b3e-a206-4999-b8b2-020f7c45eac5 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:44.453112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f36688-de97-48b0-835b-ff817c18921b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:44.453780 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f36688-de97-48b0-835b-ff817c18921b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:44.454061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f36688-de97-48b0-835b-ff817c18921b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:44.454335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f36688-de97-48b0-835b-ff817c18921b None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:44.455069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f36688-de97-48b0-835b-ff817c18921b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:44.455374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f36688-de97-48b0-835b-ff817c18921b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:44.456730 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f36688-de97-48b0-835b-ff817c18921b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:44.466986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 479/956] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jul 20 09:00:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:44.489614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ab8b5-1713-40ed-a03a-778b71c22d48 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:44.531816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5acda7-f5cc-45c1-99d6-018a61b7dcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:44.533049 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5acda7-f5cc-45c1-99d6-018a61b7dcdc None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:44.533554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5acda7-f5cc-45c1-99d6-018a61b7dcdc None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:44.534041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5acda7-f5cc-45c1-99d6-018a61b7dcdc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:44.540735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5acda7-f5cc-45c1-99d6-018a61b7dcdc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:44.542540 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5acda7-f5cc-45c1-99d6-018a61b7dcdc None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-7d5acda7-f5cc-45c1-99d6-018a61b7dcdc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:44.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:44.565562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:44.565562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:44.565562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:44.565562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:44.565562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:44.565562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:44.565562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:44.565562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:44.565562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:44.565562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:44.565562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:44.565562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:44.565562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:44.565562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:44.565562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:44.565562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:44.566490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:44.566490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:44.566490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:44.566490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:44.566490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:44.566490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:44.566490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:44.566490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:44.566490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:44.566490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:44.566490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:44.566490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:44.566490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:00:44.566490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:44.569864 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 478/957] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jul 20 09:00:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:44.576888 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f36688-de97-48b0-835b-ff817c18921b None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:44.591778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:44.689428 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f36688-de97-48b0-835b-ff817c18921b None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:44.726464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264d211-32e1-425f-993e-6e5a216f3230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:44.727529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264d211-32e1-425f-993e-6e5a216f3230 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:44.728070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264d211-32e1-425f-993e-6e5a216f3230 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:44.728555 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264d211-32e1-425f-993e-6e5a216f3230 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:44.729470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264d211-32e1-425f-993e-6e5a216f3230 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:44.729996 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264d211-32e1-425f-993e-6e5a216f3230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:44.732331 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264d211-32e1-425f-993e-6e5a216f3230 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:44.737870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 480/958] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jul 20 09:00:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:44.743700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5acda7-f5cc-45c1-99d6-018a61b7dcdc None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:44.794502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:44.854295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5acda7-f5cc-45c1-99d6-018a61b7dcdc None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:44.892336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64298215-913f-4484-9dde-09ff99d2403a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:44.893429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64298215-913f-4484-9dde-09ff99d2403a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:44.894026 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64298215-913f-4484-9dde-09ff99d2403a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:44.894520 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64298215-913f-4484-9dde-09ff99d2403a None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:44.895470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64298215-913f-4484-9dde-09ff99d2403a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:44.895984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64298215-913f-4484-9dde-09ff99d2403a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:44.897556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64298215-913f-4484-9dde-09ff99d2403a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:44.904103 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 479/959] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jul 20 09:00:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:44.910012 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264d211-32e1-425f-993e-6e5a216f3230 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:44.970906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee88dd6-55f9-4bad-9e12-123dea27834d None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:44.971554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee88dd6-55f9-4bad-9e12-123dea27834d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:44.972134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee88dd6-55f9-4bad-9e12-123dea27834d None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:44.974072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 480/960] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:44.979746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781791b5-8346-417f-b5f3-cb2708afe3b0 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:44.980037 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781791b5-8346-417f-b5f3-cb2708afe3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:44.980319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781791b5-8346-417f-b5f3-cb2708afe3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:44.993999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:44.997932 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-781791b5-8346-417f-b5f3-cb2708afe3b0 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:44.999151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-781791b5-8346-417f-b5f3-cb2708afe3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:45.021422 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264d211-32e1-425f-993e-6e5a216f3230 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:45.060890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe57973-c3bb-4236-9cc6-03a114f0837f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:45.061569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe57973-c3bb-4236-9cc6-03a114f0837f None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:45.062008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe57973-c3bb-4236-9cc6-03a114f0837f None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:45.062292 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe57973-c3bb-4236-9cc6-03a114f0837f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:45.063085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe57973-c3bb-4236-9cc6-03a114f0837f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:45.064451 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe57973-c3bb-4236-9cc6-03a114f0837f None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:45.075477 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 481/961] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:00:45.077251 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd9d26-40f7-46df-a33d-c5d92c9af65d None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:45.077599 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd9d26-40f7-46df-a33d-c5d92c9af65d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:45.077882 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd9d26-40f7-46df-a33d-c5d92c9af65d None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:45.079271 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 482/962] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:45.084581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cef21-2b3c-4a24-9c2c-7dea36f2e31d None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:45.084919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cef21-2b3c-4a24-9c2c-7dea36f2e31d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:45.085204 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cef21-2b3c-4a24-9c2c-7dea36f2e31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:45.099466 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 481/963] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jul 20 09:00:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:00:45.100526 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe57973-c3bb-4236-9cc6-03a114f0837f None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:45.110206 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-4b8cef21-2b3c-4a24-9c2c-7dea36f2e31d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:45.111474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-4b8cef21-2b3c-4a24-9c2c-7dea36f2e31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:45.152232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:45.214108 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 483/964] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:00:45.235925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe57973-c3bb-4236-9cc6-03a114f0837f None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:45.240652 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cef21-2b3c-4a24-9c2c-7dea36f2e31d None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:45.281101 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:45.296139 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a479f2e-e81f-489f-8aef-7d922e3fbe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:45.296981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a479f2e-e81f-489f-8aef-7d922e3fbe0b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:45.297264 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a479f2e-e81f-489f-8aef-7d922e3fbe0b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:45.297922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a479f2e-e81f-489f-8aef-7d922e3fbe0b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:45.310155 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a479f2e-e81f-489f-8aef-7d922e3fbe0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:45.319204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-4a479f2e-e81f-489f-8aef-7d922e3fbe0b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:00:45.319204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:45.319204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:45.319204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:45.319204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:45.319204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:45.319204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:45.319204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:45.319204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:45.319204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:45.319204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:45.319204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:45.319204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:45.319204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:45.319204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:45.319204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:45.320402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:45.320402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:45.320402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:45.320402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:45.320402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:45.320402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:45.320402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:45.320402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:45.320402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:45.320402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:45.320402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:45.320402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:45.320402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:45.320402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:45.320402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:45.320402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:45.321529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:45.321529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:45.321529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:45.321529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:45.321529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:45.321529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:45.321529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:45.321529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:45.321529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:45.321529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:45.321529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:00:45.321529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:45.321529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 482/965] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Jul 20 09:00:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:45.327595 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a479f2e-e81f-489f-8aef-7d922e3fbe0b None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:45.366175 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cef21-2b3c-4a24-9c2c-7dea36f2e31d None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:45.420088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09f1aa-8ea9-43b1-be96-f22ac111d5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:45.421243 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09f1aa-8ea9-43b1-be96-f22ac111d5fa None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:45.421535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09f1aa-8ea9-43b1-be96-f22ac111d5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:45.421823 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09f1aa-8ea9-43b1-be96-f22ac111d5fa None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:45.422995 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09f1aa-8ea9-43b1-be96-f22ac111d5fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:45.425287 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09f1aa-8ea9-43b1-be96-f22ac111d5fa None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:45.447373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a479f2e-e81f-489f-8aef-7d922e3fbe0b None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:45.448654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.sql.core [None req-3d09f1aa-8ea9-43b1-be96-f22ac111d5fa None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 09:00:45.448654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 09:00:45.448654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [parameters: {'user_id': 'd0bb7a090d1b4d3197111fb80b090acf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 09:00:45.448654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80333) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 09:00:45.457682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-3d09f1aa-8ea9-43b1-be96-f22ac111d5fa None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 09:00:45.457682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:45.457682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 09:00:45.457682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 09:00:45.457682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 09:00:45.457682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 09:00:45.457682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 09:00:45.457682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 09:00:45.457682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 09:00:45.457682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application conn.query(q) Jul 20 09:00:45.457682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 09:00:45.457682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 09:00:45.457682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 09:00:45.457682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application result.read() Jul 20 09:00:45.457682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 09:00:45.458562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application next(self.gen) Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application yield resource Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application next(self.gen) Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise self.value Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application session.commit() Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 09:00:45.460140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self.session.flush() Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application compat.raise_( Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 09:00:45.461255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 09:00:45.462327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 09:00:45.462327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 09:00:45.462327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 20 09:00:45.462327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 20 09:00:45.462327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 09:00:45.462327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 09:00:45.462327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 09:00:45.462327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 09:00:45.462327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 09:00:45.462327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 09:00:45.462327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 09:00:45.462327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 09:00:45.462327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 09:00:45.462327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 09:00:45.462327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 09:00:45.462327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception Jul 20 09:00:45.463393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 09:00:45.463393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 09:00:45.463393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 09:00:45.463393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 09:00:45.463393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 09:00:45.463393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 09:00:45.463393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 09:00:45.463393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application conn.query(q) Jul 20 09:00:45.463393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 09:00:45.463393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 09:00:45.463393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 09:00:45.463393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application result.read() Jul 20 09:00:45.463393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 09:00:45.463393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 09:00:45.463393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 09:00:45.463393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 09:00:45.464427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 09:00:45.464427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 09:00:45.464427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 09:00:45.464427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 09:00:45.464427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 09:00:45.464427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 09:00:45.464427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd0bb7a090d1b4d3197111fb80b090acf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 09:00:45.464427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 09:00:45.464427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:45.464427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:45.464427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:45.464427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:45.464427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:45.464427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:45.464427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 20 09:00:45.466061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 20 09:00:45.474448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 20 09:00:45.474448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 20 09:00:45.474448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 09:00:45.474448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 09:00:45.474448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 09:00:45.474448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:45.474448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 484/966] 10.222.0.44 () {60 vars in 1131 bytes} [Wed Jul 20 09:00:45 2022] POST /identity/v3/users => generated 160 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:45.474448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09f1aa-8ea9-43b1-be96-f22ac111d5fa None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:45.493601 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d27885-9865-40c3-85f0-4342f3906b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:45.494194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d27885-9865-40c3-85f0-4342f3906b08 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:45.494480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d27885-9865-40c3-85f0-4342f3906b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:45.494818 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d27885-9865-40c3-85f0-4342f3906b08 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:45.496044 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d27885-9865-40c3-85f0-4342f3906b08 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:45.496360 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d27885-9865-40c3-85f0-4342f3906b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:45.498157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d27885-9865-40c3-85f0-4342f3906b08 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:45.510053 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:45.514348 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 483/967] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Jul 20 09:00:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:45.519579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d27885-9865-40c3-85f0-4342f3906b08 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:45.586586 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:45.596332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09f1aa-8ea9-43b1-be96-f22ac111d5fa None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:45.632241 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9123b6b-a490-4f81-be59-b9c0b03577db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:45.633326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9123b6b-a490-4f81-be59-b9c0b03577db None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:45.633773 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9123b6b-a490-4f81-be59-b9c0b03577db None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:45.634195 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9123b6b-a490-4f81-be59-b9c0b03577db None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:45.642608 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9123b6b-a490-4f81-be59-b9c0b03577db None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:45.642795 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d27885-9865-40c3-85f0-4342f3906b08 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:45.644567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9123b6b-a490-4f81-be59-b9c0b03577db None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:45.661432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-c9123b6b-a490-4f81-be59-b9c0b03577db None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 09:00:45.661432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:45.661432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:45.661432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:45.661432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:45.661432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:45.661432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:45.661432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:45.661432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:45.661432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:45.661432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:45.661432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:45.661432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:45.661432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:45.661432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:45.661432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:45.662559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:45.663602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:45.663602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:45.663602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:45.663602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:45.663602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:45.663602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:45.663602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:45.663602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:45.663602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 09:00:45.663602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:45.670926 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 485/968] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:00:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:45.673944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9123b6b-a490-4f81-be59-b9c0b03577db None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:45.683113 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdf767-6eb3-4f00-a84f-0bb12b709b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:45.684793 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdf767-6eb3-4f00-a84f-0bb12b709b0d None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:45.685284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdf767-6eb3-4f00-a84f-0bb12b709b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:45.686219 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdf767-6eb3-4f00-a84f-0bb12b709b0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:45.700891 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdf767-6eb3-4f00-a84f-0bb12b709b0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:45.703383 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdf767-6eb3-4f00-a84f-0bb12b709b0d None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:45.727284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-eafdf767-6eb3-4f00-a84f-0bb12b709b0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:45.727284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:45.727284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:45.727284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:45.727284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:45.727284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:45.727284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:45.727284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:45.727284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:45.727284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:45.727284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:45.727284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:45.727284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:45.727284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:45.727284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:45.728239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:45.729144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:45.729144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:45.729144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:45.729144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:45.729144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 484/969] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:45.736432 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdf767-6eb3-4f00-a84f-0bb12b709b0d None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:45.770900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:45.788397 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9123b6b-a490-4f81-be59-b9c0b03577db None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:45.832646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4d1b9-9061-4997-b319-c53cf5a829d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:45.833653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4d1b9-9061-4997-b319-c53cf5a829d1 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:45.834184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4d1b9-9061-4997-b319-c53cf5a829d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:45.834786 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4d1b9-9061-4997-b319-c53cf5a829d1 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:45.835913 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4d1b9-9061-4997-b319-c53cf5a829d1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:45.836425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4d1b9-9061-4997-b319-c53cf5a829d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:45.838238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4d1b9-9061-4997-b319-c53cf5a829d1 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:45.844712 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdf767-6eb3-4f00-a84f-0bb12b709b0d None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:45.847078 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:45.886818 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 486/970] 10.222.0.44 () {56 vars in 1101 bytes} [Wed Jul 20 09:00:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:45.896266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de51e1d-ad0e-49c4-adc4-64752bc5824d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:45.896964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de51e1d-ad0e-49c4-adc4-64752bc5824d None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:45.897252 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de51e1d-ad0e-49c4-adc4-64752bc5824d None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:45.897532 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de51e1d-ad0e-49c4-adc4-64752bc5824d None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:45.898280 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de51e1d-ad0e-49c4-adc4-64752bc5824d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:45.898593 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de51e1d-ad0e-49c4-adc4-64752bc5824d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:45.900057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de51e1d-ad0e-49c4-adc4-64752bc5824d None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:45.914107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 485/971] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:45.920035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4d1b9-9061-4997-b319-c53cf5a829d1 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:46.031638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4d1b9-9061-4997-b319-c53cf5a829d1 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:46.070291 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab33ce9-2fab-450a-9e24-6910e69c8f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:46.071149 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab33ce9-2fab-450a-9e24-6910e69c8f40 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:46.071434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab33ce9-2fab-450a-9e24-6910e69c8f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:46.071714 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab33ce9-2fab-450a-9e24-6910e69c8f40 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:46.081058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab33ce9-2fab-450a-9e24-6910e69c8f40 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:46.082497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab33ce9-2fab-450a-9e24-6910e69c8f40 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:46.097777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-cab33ce9-2fab-450a-9e24-6910e69c8f40 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:00:46.097777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:46.097777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:46.097777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:46.097777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:46.097777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:46.097777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:46.097777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:46.097777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:46.097777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:46.097777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:46.097777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:46.097777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:46.097777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:46.097777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:46.097777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:46.098810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:46.100636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:46.100636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:46.100636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:46.100636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:46.100636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:46.100636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:46.100636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:46.100636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:46.100636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 09:00:46.100636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:46.100636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 487/972] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jul 20 09:00:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:46.105125 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de51e1d-ad0e-49c4-adc4-64752bc5824d None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:46.217761 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de51e1d-ad0e-49c4-adc4-64752bc5824d None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:46.253015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1a0d0-aaff-4714-97d1-acdce9e61fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:46.253692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1a0d0-aaff-4714-97d1-acdce9e61fca None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:46.253980 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1a0d0-aaff-4714-97d1-acdce9e61fca None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:46.254264 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1a0d0-aaff-4714-97d1-acdce9e61fca None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:46.255131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1a0d0-aaff-4714-97d1-acdce9e61fca None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:46.255447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1a0d0-aaff-4714-97d1-acdce9e61fca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:46.256885 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1a0d0-aaff-4714-97d1-acdce9e61fca None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:46.275131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 486/973] 10.222.0.44 () {56 vars in 1149 bytes} [Wed Jul 20 09:00:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:46.280678 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab33ce9-2fab-450a-9e24-6910e69c8f40 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:46.284603 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:46.388714 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab33ce9-2fab-450a-9e24-6910e69c8f40 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:46.424497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a627400-63e7-4823-ab3e-f5a2b0759ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:46.425151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a627400-63e7-4823-ab3e-f5a2b0759ebd None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:46.425435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a627400-63e7-4823-ab3e-f5a2b0759ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:46.425711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a627400-63e7-4823-ab3e-f5a2b0759ebd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:46.495012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a627400-63e7-4823-ab3e-f5a2b0759ebd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:46.496679 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a627400-63e7-4823-ab3e-f5a2b0759ebd None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:46.499590 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:46.607727 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d911ca-723f-4d0e-9222-dc11e8dc8a9f None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:46.608047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d911ca-723f-4d0e-9222-dc11e8dc8a9f None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:46.608288 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d911ca-723f-4d0e-9222-dc11e8dc8a9f None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:46.613063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 488/974] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:00:46 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:00:46.613184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 487/975] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:46.614101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a627400-63e7-4823-ab3e-f5a2b0759ebd None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:46.616297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbaab46-cbe5-40de-b4f3-730c40b17d54 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:46.616585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbaab46-cbe5-40de-b4f3-730c40b17d54 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:46.616869 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbaab46-cbe5-40de-b4f3-730c40b17d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:46.634560 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-8cbaab46-cbe5-40de-b4f3-730c40b17d54 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:46.636198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8cbaab46-cbe5-40de-b4f3-730c40b17d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:46.653723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:46.708514 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 488/976] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:00:46.723861 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a627400-63e7-4823-ab3e-f5a2b0759ebd None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:46.726954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:46.730722 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbaab46-cbe5-40de-b4f3-730c40b17d54 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:46.763513 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf670b9a-0fa9-4acc-877d-95115a7dc349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:46.764412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf670b9a-0fa9-4acc-877d-95115a7dc349 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:46.764692 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf670b9a-0fa9-4acc-877d-95115a7dc349 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:46.764963 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf670b9a-0fa9-4acc-877d-95115a7dc349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:46.776477 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf670b9a-0fa9-4acc-877d-95115a7dc349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:46.778220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf670b9a-0fa9-4acc-877d-95115a7dc349 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:46.798254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-bf670b9a-0fa9-4acc-877d-95115a7dc349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:46.798254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:46.798254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:46.798254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:46.798254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:46.798254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:46.798254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:46.798254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:46.798254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:46.798254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:46.798254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:46.798254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:46.798254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:46.798254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:46.798254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:46.799393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:46.800626 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:46.800626 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:46.800626 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:46.800626 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:46.800626 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 489/977] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:46.805050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf670b9a-0fa9-4acc-877d-95115a7dc349 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:46.836717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbaab46-cbe5-40de-b4f3-730c40b17d54 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:46.876327 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b873835-ef10-4bf4-a0ff-56525a6ab264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:46.877103 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b873835-ef10-4bf4-a0ff-56525a6ab264 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:46.877387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b873835-ef10-4bf4-a0ff-56525a6ab264 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:46.877666 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b873835-ef10-4bf4-a0ff-56525a6ab264 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:46.878386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b873835-ef10-4bf4-a0ff-56525a6ab264 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:46.879846 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b873835-ef10-4bf4-a0ff-56525a6ab264 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:46.898851 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 489/978] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jul 20 09:00:46 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:00:46.908699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf670b9a-0fa9-4acc-877d-95115a7dc349 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:46.955403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597eae98-92bd-4789-8600-82a810ca816d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:46.956092 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597eae98-92bd-4789-8600-82a810ca816d None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:46.956376 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597eae98-92bd-4789-8600-82a810ca816d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:46.956653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597eae98-92bd-4789-8600-82a810ca816d None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:46.957390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597eae98-92bd-4789-8600-82a810ca816d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:46.957702 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597eae98-92bd-4789-8600-82a810ca816d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:46.959117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597eae98-92bd-4789-8600-82a810ca816d None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:46.976568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 490/979] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:46.978466 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b873835-ef10-4bf4-a0ff-56525a6ab264 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:46.995187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:47.093259 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b873835-ef10-4bf4-a0ff-56525a6ab264 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:47.128449 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775bc09-411c-493b-a591-84993bf2a5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:47.129239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775bc09-411c-493b-a591-84993bf2a5ab None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:47.129517 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775bc09-411c-493b-a591-84993bf2a5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:47.129787 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775bc09-411c-493b-a591-84993bf2a5ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:47.136553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9775bc09-411c-493b-a591-84993bf2a5ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:47.138022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9775bc09-411c-493b-a591-84993bf2a5ab None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:47.151925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-9775bc09-411c-493b-a591-84993bf2a5ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:47.151925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:47.151925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:47.151925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:47.151925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:47.151925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:47.151925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:47.151925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:47.151925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:47.151925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:47.151925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:47.151925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:47.151925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:47.151925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:47.151925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:47.153386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:47.159115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:47.159115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:47.159115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:47.159115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:47.159115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:47.159115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:47.159115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:47.159115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:47.159115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 490/980] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:47.161109 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597eae98-92bd-4789-8600-82a810ca816d None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:47.177829 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:47.264463 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597eae98-92bd-4789-8600-82a810ca816d None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:47.299325 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d6023-ea48-4021-bbec-1f5cb2684445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:47.299985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d6023-ea48-4021-bbec-1f5cb2684445 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:47.300267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d6023-ea48-4021-bbec-1f5cb2684445 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:47.300542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d6023-ea48-4021-bbec-1f5cb2684445 None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:47.301322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d6023-ea48-4021-bbec-1f5cb2684445 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:47.301632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d6023-ea48-4021-bbec-1f5cb2684445 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:47.303058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d6023-ea48-4021-bbec-1f5cb2684445 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:47.315777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 491/981] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:47.321697 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775bc09-411c-493b-a591-84993bf2a5ab None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:47.330630 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b74f6-3fd2-46b2-a2ea-d84fe1b084d0 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:47.330966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b74f6-3fd2-46b2-a2ea-d84fe1b084d0 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:47.331248 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b74f6-3fd2-46b2-a2ea-d84fe1b084d0 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:47.332622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 492/982] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:47.337536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc8743-a807-4ea1-9a1b-8518afa5843d None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:47.337821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc8743-a807-4ea1-9a1b-8518afa5843d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:47.338103 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc8743-a807-4ea1-9a1b-8518afa5843d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:47.355812 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-0ddc8743-a807-4ea1-9a1b-8518afa5843d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:47.358104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-0ddc8743-a807-4ea1-9a1b-8518afa5843d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:47.405678 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:47.437198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775bc09-411c-493b-a591-84993bf2a5ab None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:47.441058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 493/983] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:00:47.464859 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc8743-a807-4ea1-9a1b-8518afa5843d None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:47.467693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:47.478406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44e8be-1198-4631-813d-459b8b4c47d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:47.479418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44e8be-1198-4631-813d-459b8b4c47d6 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:47.479939 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44e8be-1198-4631-813d-459b8b4c47d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:47.480142 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44e8be-1198-4631-813d-459b8b4c47d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:47.481064 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44e8be-1198-4631-813d-459b8b4c47d6 None admin] RBAC: Adding query filter params (role.id=a2636fed60634c3997e3c9213a818395, scope.project.id=a498284274454d25b1b118a88996e522, user.id=6b8db4a7ee284b1aa96d4517dd381e71) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:47.481381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44e8be-1198-4631-813d-459b8b4c47d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:47.482832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44e8be-1198-4631-813d-459b8b4c47d6 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:47.489267 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 491/984] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:00:47 2022] GET /identity/v3/role_assignments?user.id=6b8db4a7ee284b1aa96d4517dd381e71&scope.project.id=a498284274454d25b1b118a88996e522&role.id=a2636fed60634c3997e3c9213a818395 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:47.586750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc8743-a807-4ea1-9a1b-8518afa5843d None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:47.626250 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ec840-72d4-4288-8b5e-49f122f63563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:47.627382 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ec840-72d4-4288-8b5e-49f122f63563 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:47.627822 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ec840-72d4-4288-8b5e-49f122f63563 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:47.628275 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ec840-72d4-4288-8b5e-49f122f63563 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:47.629103 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ec840-72d4-4288-8b5e-49f122f63563 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:47.630630 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ec840-72d4-4288-8b5e-49f122f63563 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-b64ec840-72d4-4288-8b5e-49f122f63563 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:47.644516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:47.645568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:47.645568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:47.645568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:47.645568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:47.645568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:47.645568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:47.645568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:47.645568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:47.645568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:47.645568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:47.645568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:47.645568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:47.645568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:47.645568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:47.645568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:47.645568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:47.646418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:47.646418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:47.646418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:47.646418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:47.646418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:47.646418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:47.646418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:47.646418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:47.646418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:47.646418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:47.646418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:47.646418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:47.646418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:00:47.646418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:47.646418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 494/985] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jul 20 09:00:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:47.651640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44e8be-1198-4631-813d-459b8b4c47d6 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:47.755548 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44e8be-1198-4631-813d-459b8b4c47d6 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:47.788507 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b8669-3841-4668-861a-c2c75dbafb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:47.789164 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8669-3841-4668-861a-c2c75dbafb3e None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:47.789445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8669-3841-4668-861a-c2c75dbafb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:47.789722 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8669-3841-4668-861a-c2c75dbafb3e None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:47.790483 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b8669-3841-4668-861a-c2c75dbafb3e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:47.790824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b8669-3841-4668-861a-c2c75dbafb3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:47.792230 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b8669-3841-4668-861a-c2c75dbafb3e None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:47.794550 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:47.798714 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 492/986] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jul 20 09:00:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:47.804453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ec840-72d4-4288-8b5e-49f122f63563 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:47.870331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:47.906877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ec840-72d4-4288-8b5e-49f122f63563 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:47.940048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7c722-f3f8-4919-8ef1-f96dad389fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:47.940757 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7c722-f3f8-4919-8ef1-f96dad389fce None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:47.940981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7c722-f3f8-4919-8ef1-f96dad389fce None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:47.941257 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7c722-f3f8-4919-8ef1-f96dad389fce None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:47.941960 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7c722-f3f8-4919-8ef1-f96dad389fce None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:47.942265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7c722-f3f8-4919-8ef1-f96dad389fce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:47.943647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f7c722-f3f8-4919-8ef1-f96dad389fce None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:47.950773 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 495/987] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jul 20 09:00:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:47.955100 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b8669-3841-4668-861a-c2c75dbafb3e None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:48.060560 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b8669-3841-4668-861a-c2c75dbafb3e None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:48.095269 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6dbf8-def7-438b-9d08-83c5f7e7749b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:48.095927 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6dbf8-def7-438b-9d08-83c5f7e7749b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:48.096208 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6dbf8-def7-438b-9d08-83c5f7e7749b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:48.096483 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6dbf8-def7-438b-9d08-83c5f7e7749b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:48.097277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6dbf8-def7-438b-9d08-83c5f7e7749b None admin] RBAC: Adding query filter params (interface=admin, service_id=7d362264b4b84421af8604a13b3c6722, region_id=RegionOne) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:48.097587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6dbf8-def7-438b-9d08-83c5f7e7749b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:48.098967 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6dbf8-def7-438b-9d08-83c5f7e7749b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:48.108981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 493/988] 10.222.0.44 () {56 vars in 1249 bytes} [Wed Jul 20 09:00:47 2022] GET /identity/v3/endpoints?service_id=7d362264b4b84421af8604a13b3c6722&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:48.110280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7c722-f3f8-4919-8ef1-f96dad389fce None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:48.213573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7c722-f3f8-4919-8ef1-f96dad389fce None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:48.242260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:48.249137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c30ddc-a7a7-44fc-bdb6-06c3dbeaf8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:48.249802 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c30ddc-a7a7-44fc-bdb6-06c3dbeaf8fe None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:48.250083 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c30ddc-a7a7-44fc-bdb6-06c3dbeaf8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:48.250357 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c30ddc-a7a7-44fc-bdb6-06c3dbeaf8fe None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:48.251076 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c30ddc-a7a7-44fc-bdb6-06c3dbeaf8fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:48.252428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c30ddc-a7a7-44fc-bdb6-06c3dbeaf8fe None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:48.263739 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 496/989] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jul 20 09:00:48 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:00:48.982347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95290cf-4211-4578-883c-8973a0a343bc None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:48.982792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95290cf-4211-4578-883c-8973a0a343bc None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:48.982973 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95290cf-4211-4578-883c-8973a0a343bc None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:48.984506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 494/990] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:48.989486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8deb07e-cae9-496f-8365-2cf3f6797fe3 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:48.989920 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8deb07e-cae9-496f-8365-2cf3f6797fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:48.990099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8deb07e-cae9-496f-8365-2cf3f6797fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:49.007043 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-f8deb07e-cae9-496f-8365-2cf3f6797fe3 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:49.008214 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f8deb07e-cae9-496f-8365-2cf3f6797fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:49.079002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 497/991] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:49.101338 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95290cf-4211-4578-883c-8973a0a343bc None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:49.166040 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:49.207888 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95290cf-4211-4578-883c-8973a0a343bc None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:49.243009 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9906ac-4fa0-445d-ab70-6094f3dbd1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:49.244166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9906ac-4fa0-445d-ab70-6094f3dbd1b4 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:49.244687 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9906ac-4fa0-445d-ab70-6094f3dbd1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:49.245263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9906ac-4fa0-445d-ab70-6094f3dbd1b4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:49.246127 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9906ac-4fa0-445d-ab70-6094f3dbd1b4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:49.247822 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9906ac-4fa0-445d-ab70-6094f3dbd1b4 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-aa9906ac-4fa0-445d-ab70-6094f3dbd1b4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:49.260231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:49.261248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:49.261248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:49.261248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:49.261248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:49.261248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:49.261248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:49.261248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:49.261248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:49.261248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:49.261248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:49.261248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:49.261248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:49.261248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:49.261248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:49.261248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:49.261248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:49.262655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:49.262655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:49.262655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:49.262655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:49.262655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:49.262655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:49.262655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:49.262655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:49.262655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:49.262655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:49.262655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:49.262655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:49.262655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:00:49.262655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:49.264196 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 495/992] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:49.270542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8deb07e-cae9-496f-8365-2cf3f6797fe3 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:49.376613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8deb07e-cae9-496f-8365-2cf3f6797fe3 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:49.401193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:49.414532 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48341a-125a-467f-b68e-1f7af540edf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:49.415293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48341a-125a-467f-b68e-1f7af540edf2 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:49.415596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48341a-125a-467f-b68e-1f7af540edf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:49.415881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48341a-125a-467f-b68e-1f7af540edf2 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:49.416616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48341a-125a-467f-b68e-1f7af540edf2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:49.416932 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48341a-125a-467f-b68e-1f7af540edf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:49.418370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48341a-125a-467f-b68e-1f7af540edf2 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:49.428184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 498/993] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jul 20 09:00:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:49.429312 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9906ac-4fa0-445d-ab70-6094f3dbd1b4 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:49.533772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9906ac-4fa0-445d-ab70-6094f3dbd1b4 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:49.568319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:49.578221 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e41c0-f69c-4b3f-b444-a2f8febe0c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:49.578928 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e41c0-f69c-4b3f-b444-a2f8febe0c86 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:49.579220 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e41c0-f69c-4b3f-b444-a2f8febe0c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:49.579502 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e41c0-f69c-4b3f-b444-a2f8febe0c86 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:49.580216 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e41c0-f69c-4b3f-b444-a2f8febe0c86 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:49.580527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e41c0-f69c-4b3f-b444-a2f8febe0c86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:49.581963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e41c0-f69c-4b3f-b444-a2f8febe0c86 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:49.591817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 496/994] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jul 20 09:00:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:49.593047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48341a-125a-467f-b68e-1f7af540edf2 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:49.622072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47ef41-bba3-4ced-bde2-f2ee85223eae None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:49.622366 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47ef41-bba3-4ced-bde2-f2ee85223eae None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:49.622647 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47ef41-bba3-4ced-bde2-f2ee85223eae None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:49.624166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 497/995] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:49.628758 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb91da6-cee6-4e1c-a284-7cb45b274343 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:49.629052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb91da6-cee6-4e1c-a284-7cb45b274343 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:49.629341 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb91da6-cee6-4e1c-a284-7cb45b274343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:49.646207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-1eb91da6-cee6-4e1c-a284-7cb45b274343 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:49.647426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-1eb91da6-cee6-4e1c-a284-7cb45b274343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:49.695796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48341a-125a-467f-b68e-1f7af540edf2 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:49.722815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 498/996] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:00:49.734390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5929a9-2c2e-4c85-a81c-7ace07d1569a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:49.735074 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5929a9-2c2e-4c85-a81c-7ace07d1569a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:49.735355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5929a9-2c2e-4c85-a81c-7ace07d1569a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:49.735628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5929a9-2c2e-4c85-a81c-7ace07d1569a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:49.736411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5929a9-2c2e-4c85-a81c-7ace07d1569a None admin] RBAC: Adding query filter params (interface=public, service_id=0e8dbe633a7b4638982e3f135e01ce22, region_id=RegionOne) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:49.736721 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5929a9-2c2e-4c85-a81c-7ace07d1569a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:49.738067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5929a9-2c2e-4c85-a81c-7ace07d1569a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:49.743217 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 499/997] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Jul 20 09:00:49 2022] GET /identity/v3/endpoints?service_id=0e8dbe633a7b4638982e3f135e01ce22&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:49.744321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb91da6-cee6-4e1c-a284-7cb45b274343 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:49.752191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5929a9-2c2e-4c85-a81c-7ace07d1569a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:49.807828 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:49.857762 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb91da6-cee6-4e1c-a284-7cb45b274343 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:49.858448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5929a9-2c2e-4c85-a81c-7ace07d1569a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:49.880837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:49.893744 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83365cc8-545f-4a62-9611-26b77a7ec6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:49.894590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83365cc8-545f-4a62-9611-26b77a7ec6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:49.894846 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83365cc8-545f-4a62-9611-26b77a7ec6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:49.895122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83365cc8-545f-4a62-9611-26b77a7ec6e6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:49.895676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ebec1-2496-43b5-a275-304f12023489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:49.895793 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83365cc8-545f-4a62-9611-26b77a7ec6e6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:49.896345 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ebec1-2496-43b5-a275-304f12023489 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:49.896630 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ebec1-2496-43b5-a275-304f12023489 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:49.896913 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ebec1-2496-43b5-a275-304f12023489 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:49.897296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83365cc8-545f-4a62-9611-26b77a7ec6e6 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:49.897683 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ebec1-2496-43b5-a275-304f12023489 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:49.899092 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ebec1-2496-43b5-a275-304f12023489 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:49.905980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 500/998] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jul 20 09:00:49 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-83365cc8-545f-4a62-9611-26b77a7ec6e6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:49.909480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:49.910481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:49.910481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:49.910481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:49.910481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:49.910481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:49.910481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:49.910481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:49.910481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:49.910481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:49.910481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:49.910481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:49.910481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:49.910481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:49.910481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:49.910481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:49.910481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:49.912102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:49.912102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:49.912102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:49.912102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:49.912102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:49.912102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:49.912102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:49.912102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:49.912102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:49.912102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:49.912102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:49.912102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:49.912102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:00:49.912102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:49.912102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 499/999] 10.222.0.44 () {56 vars in 1105 bytes} [Wed Jul 20 09:00:49 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:49.922817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ebec1-2496-43b5-a275-304f12023489 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:50.033579 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ebec1-2496-43b5-a275-304f12023489 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:50.052425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:50.073384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a4b07-fb5b-4b1d-bd58-c8111c128477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:50.073980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a4b07-fb5b-4b1d-bd58-c8111c128477 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:50.074254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a4b07-fb5b-4b1d-bd58-c8111c128477 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:50.075071 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a4b07-fb5b-4b1d-bd58-c8111c128477 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:50.075781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a4b07-fb5b-4b1d-bd58-c8111c128477 None admin] RBAC: Adding query filter params (name=image) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:50.076086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a4b07-fb5b-4b1d-bd58-c8111c128477 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:50.077438 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a4b07-fb5b-4b1d-bd58-c8111c128477 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:50.083347 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 501/1000] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:50.089191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83365cc8-545f-4a62-9611-26b77a7ec6e6 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:50.193040 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83365cc8-545f-4a62-9611-26b77a7ec6e6 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:50.227301 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36a249-1bd8-4a93-ac1b-b4e3c9387906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:50.228042 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36a249-1bd8-4a93-ac1b-b4e3c9387906 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:50.228336 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36a249-1bd8-4a93-ac1b-b4e3c9387906 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:50.228622 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36a249-1bd8-4a93-ac1b-b4e3c9387906 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:50.229335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36a249-1bd8-4a93-ac1b-b4e3c9387906 None admin] RBAC: Adding query filter params (type=image) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:50.229652 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36a249-1bd8-4a93-ac1b-b4e3c9387906 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:50.231671 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36a249-1bd8-4a93-ac1b-b4e3c9387906 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:50.236453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 500/1001] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:50.359463 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabcc708-f346-460f-a875-69f7a7f64313 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:50.359804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabcc708-f346-460f-a875-69f7a7f64313 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:50.360029 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabcc708-f346-460f-a875-69f7a7f64313 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:50.361668 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 502/1002] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:50.367144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cee886-11f3-4d5d-8f99-f8787221b8b4 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:50.367440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cee886-11f3-4d5d-8f99-f8787221b8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:50.367723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cee886-11f3-4d5d-8f99-f8787221b8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:50.386736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-39cee886-11f3-4d5d-8f99-f8787221b8b4 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:50.388044 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-39cee886-11f3-4d5d-8f99-f8787221b8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:50.438818 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:50.461260 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 501/1003] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:50.486182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabcc708-f346-460f-a875-69f7a7f64313 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:50.601318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabcc708-f346-460f-a875-69f7a7f64313 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:50.635943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf27a2e-6e05-446a-aac7-766bc0fddd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:50.636667 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf27a2e-6e05-446a-aac7-766bc0fddd15 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:50.637649 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf27a2e-6e05-446a-aac7-766bc0fddd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:50.637815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf27a2e-6e05-446a-aac7-766bc0fddd15 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:50.638406 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf27a2e-6e05-446a-aac7-766bc0fddd15 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:50.639826 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf27a2e-6e05-446a-aac7-766bc0fddd15 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-0cf27a2e-6e05-446a-aac7-766bc0fddd15 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:50.651857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:50.658905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:50.658905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:50.658905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:50.658905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:50.658905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:50.658905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:50.658905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:50.658905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:50.658905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:50.658905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:50.658905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:50.658905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:50.658905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:50.658905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:50.658905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:50.658905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:50.659837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:50.659837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:50.659837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:50.659837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:50.659837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:50.659837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:50.659837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:50.659837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:50.659837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:50.659837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:50.659837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:50.659837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:50.659837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 09:00:50.659837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:50.659837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 503/1004] 10.222.0.44 () {56 vars in 1126 bytes} [Wed Jul 20 09:00:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:50.662163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cee886-11f3-4d5d-8f99-f8787221b8b4 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:50.763253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cee886-11f3-4d5d-8f99-f8787221b8b4 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:50.797576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902278f8-b3e2-4639-bfe8-12727268d3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:50.798291 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902278f8-b3e2-4639-bfe8-12727268d3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:50.798575 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902278f8-b3e2-4639-bfe8-12727268d3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:50.798906 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902278f8-b3e2-4639-bfe8-12727268d3b9 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:50.799614 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902278f8-b3e2-4639-bfe8-12727268d3b9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:50.799914 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902278f8-b3e2-4639-bfe8-12727268d3b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:50.801264 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902278f8-b3e2-4639-bfe8-12727268d3b9 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:50.810416 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 502/1005] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jul 20 09:00:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:50.811654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf27a2e-6e05-446a-aac7-766bc0fddd15 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:50.860529 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:50.917879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf27a2e-6e05-446a-aac7-766bc0fddd15 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:50.952507 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ee27f-8793-42ca-a125-11135e66eb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:50.953173 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ee27f-8793-42ca-a125-11135e66eb8f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:50.953451 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ee27f-8793-42ca-a125-11135e66eb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:50.953729 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ee27f-8793-42ca-a125-11135e66eb8f None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:50.954432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ee27f-8793-42ca-a125-11135e66eb8f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:50.954774 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ee27f-8793-42ca-a125-11135e66eb8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:50.956127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ee27f-8793-42ca-a125-11135e66eb8f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:50.965675 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 504/1006] 10.222.0.44 () {56 vars in 1122 bytes} [Wed Jul 20 09:00:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:50.967344 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902278f8-b3e2-4639-bfe8-12727268d3b9 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:51.015547 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:51.073377 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902278f8-b3e2-4639-bfe8-12727268d3b9 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:51.107734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734dc4d-c218-4dcc-ad78-73cdd6d17cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:51.108400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734dc4d-c218-4dcc-ad78-73cdd6d17cfb None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:51.108688 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734dc4d-c218-4dcc-ad78-73cdd6d17cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:51.108969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734dc4d-c218-4dcc-ad78-73cdd6d17cfb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:51.109712 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734dc4d-c218-4dcc-ad78-73cdd6d17cfb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:51.111123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734dc4d-c218-4dcc-ad78-73cdd6d17cfb None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:51.127680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 503/1007] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jul 20 09:00:50 2022] POST /identity/v3/endpoints => generated 325 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:00:51.963579 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01478cd6-fb2c-40fb-8833-f79a10d023c9 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:51.964102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01478cd6-fb2c-40fb-8833-f79a10d023c9 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:51.964181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01478cd6-fb2c-40fb-8833-f79a10d023c9 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:51.966095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 505/1008] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:51.971649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172ed31-8afa-400b-8f34-d929c091d28b None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:51.972053 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172ed31-8afa-400b-8f34-d929c091d28b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:51.972221 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172ed31-8afa-400b-8f34-d929c091d28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:51.989509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-9172ed31-8afa-400b-8f34-d929c091d28b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:51.990701 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-9172ed31-8afa-400b-8f34-d929c091d28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:52.065838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 504/1009] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:52.088251 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01478cd6-fb2c-40fb-8833-f79a10d023c9 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:52.192595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01478cd6-fb2c-40fb-8833-f79a10d023c9 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:52.202565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:52.228673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4e636-910a-4cf4-bf71-279d87422492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:52.229388 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4e636-910a-4cf4-bf71-279d87422492 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:52.229668 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4e636-910a-4cf4-bf71-279d87422492 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:52.229944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4e636-910a-4cf4-bf71-279d87422492 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:52.231158 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4e636-910a-4cf4-bf71-279d87422492 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:52.232548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4e636-910a-4cf4-bf71-279d87422492 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-a9a4e636-910a-4cf4-bf71-279d87422492 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:52.244139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:52.245155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:52.245155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:52.245155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:52.245155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:52.245155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:52.245155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:52.245155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:52.245155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:52.245155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:52.245155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:52.245155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:52.245155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:52.245155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:52.245155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:52.245155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:52.245155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:52.246016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:52.246016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:52.246016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:52.246016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:52.246016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:52.246016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:52.246016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:52.246016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:52.246016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:52.246016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:52.246016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:52.246016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:52.246016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 09:00:52.246016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:52.251245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 506/1010] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:52.252344 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172ed31-8afa-400b-8f34-d929c091d28b None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:52.283661 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:52.287678 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f386a-2844-41f5-8972-e5ab6e947866 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:52.287971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f386a-2844-41f5-8972-e5ab6e947866 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:52.288254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f386a-2844-41f5-8972-e5ab6e947866 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:52.289646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 507/1011] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:52.295441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb7022-3764-49f8-9dcb-a1417eb1ef16 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:52.295732 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb7022-3764-49f8-9dcb-a1417eb1ef16 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:52.296015 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb7022-3764-49f8-9dcb-a1417eb1ef16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:52.313326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-decb7022-3764-49f8-9dcb-a1417eb1ef16 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:52.314516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-decb7022-3764-49f8-9dcb-a1417eb1ef16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:52.360726 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172ed31-8afa-400b-8f34-d929c091d28b None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:52.386726 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 508/1012] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:52 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:52.400162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b0ae2-576f-46f4-b342-4920ca418dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:52.400832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b0ae2-576f-46f4-b342-4920ca418dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:52.401119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b0ae2-576f-46f4-b342-4920ca418dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:52.401400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b0ae2-576f-46f4-b342-4920ca418dc9 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:52.402111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b0ae2-576f-46f4-b342-4920ca418dc9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:52.402424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b0ae2-576f-46f4-b342-4920ca418dc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:52.404580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b0ae2-576f-46f4-b342-4920ca418dc9 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:52.408344 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb7022-3764-49f8-9dcb-a1417eb1ef16 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:52.414237 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 505/1013] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jul 20 09:00:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:52.414996 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b0ae2-576f-46f4-b342-4920ca418dc9 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:52.509879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb7022-3764-49f8-9dcb-a1417eb1ef16 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:52.511300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:52.520585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:52.528842 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b0ae2-576f-46f4-b342-4920ca418dc9 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:52.546295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f44e34-1265-4ec9-b8d7-0e0518684704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:52.546988 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f44e34-1265-4ec9-b8d7-0e0518684704 None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:52.547274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f44e34-1265-4ec9-b8d7-0e0518684704 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:52.547547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f44e34-1265-4ec9-b8d7-0e0518684704 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:52.548266 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f44e34-1265-4ec9-b8d7-0e0518684704 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:52.549613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f44e34-1265-4ec9-b8d7-0e0518684704 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:52.564990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 509/1014] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jul 20 09:00:52 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:00:52.588610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf095611-7948-4bb2-ba17-3c65fc4d1b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:52.589303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf095611-7948-4bb2-ba17-3c65fc4d1b47 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:52.589595 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf095611-7948-4bb2-ba17-3c65fc4d1b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:52.589876 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf095611-7948-4bb2-ba17-3c65fc4d1b47 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:52.591191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf095611-7948-4bb2-ba17-3c65fc4d1b47 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:52.591699 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf095611-7948-4bb2-ba17-3c65fc4d1b47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:52.593126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf095611-7948-4bb2-ba17-3c65fc4d1b47 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:52.599133 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 506/1015] 10.222.0.44 () {56 vars in 1112 bytes} [Wed Jul 20 09:00:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:52.604574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f44e34-1265-4ec9-b8d7-0e0518684704 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:52.719073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f44e34-1265-4ec9-b8d7-0e0518684704 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:52.754868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f085d12-ea5e-44ff-ba0d-1c2c4e1d0273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:52.755530 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f085d12-ea5e-44ff-ba0d-1c2c4e1d0273 None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:52.755814 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f085d12-ea5e-44ff-ba0d-1c2c4e1d0273 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:52.756090 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f085d12-ea5e-44ff-ba0d-1c2c4e1d0273 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:52.756837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f085d12-ea5e-44ff-ba0d-1c2c4e1d0273 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:52.758190 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f085d12-ea5e-44ff-ba0d-1c2c4e1d0273 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:52.779012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 510/1016] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jul 20 09:00:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:00:53.204138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcbfc9-1869-4ca5-94de-61753ab3222c None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:53.204665 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcbfc9-1869-4ca5-94de-61753ab3222c None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:53.204765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcbfc9-1869-4ca5-94de-61753ab3222c None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:53.206226 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 507/1017] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:53.211305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe16f33-4afd-4d6f-ade2-0c99aded9630 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:53.211727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe16f33-4afd-4d6f-ade2-0c99aded9630 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:53.211891 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe16f33-4afd-4d6f-ade2-0c99aded9630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:53.228719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-bfe16f33-4afd-4d6f-ade2-0c99aded9630 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:53.229914 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-bfe16f33-4afd-4d6f-ade2-0c99aded9630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:53.243793 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:53.307414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 511/1018] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:53.331953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcbfc9-1869-4ca5-94de-61753ab3222c None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:53.443639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcbfc9-1869-4ca5-94de-61753ab3222c None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:53.478289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899abbcf-5729-48a1-91c0-e8d6d6b195c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:53.479026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899abbcf-5729-48a1-91c0-e8d6d6b195c3 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:53.479315 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899abbcf-5729-48a1-91c0-e8d6d6b195c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:53.479590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899abbcf-5729-48a1-91c0-e8d6d6b195c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:53.481543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899abbcf-5729-48a1-91c0-e8d6d6b195c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:53.483120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899abbcf-5729-48a1-91c0-e8d6d6b195c3 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:53.484880 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 508/1019] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:53.490704 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe16f33-4afd-4d6f-ade2-0c99aded9630 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:53.598821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe16f33-4afd-4d6f-ade2-0c99aded9630 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:53.633357 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99de01-2413-4390-acd4-9c888475a893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:53.634079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99de01-2413-4390-acd4-9c888475a893 None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:53.634367 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99de01-2413-4390-acd4-9c888475a893 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:53.634650 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99de01-2413-4390-acd4-9c888475a893 None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:53.635398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99de01-2413-4390-acd4-9c888475a893 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:53.637353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99de01-2413-4390-acd4-9c888475a893 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:53.652184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:53.674768 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 512/1020] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jul 20 09:00:53 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:00:54.657256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03f3f0-86c5-41cb-90e1-022fa33e5194 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:54.657769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03f3f0-86c5-41cb-90e1-022fa33e5194 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:54.657847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03f3f0-86c5-41cb-90e1-022fa33e5194 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:54.659240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 509/1021] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:54.664236 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac902de-5462-4a39-be80-f588a9903681 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:54.664609 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac902de-5462-4a39-be80-f588a9903681 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:54.664788 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac902de-5462-4a39-be80-f588a9903681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:54.681695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-bac902de-5462-4a39-be80-f588a9903681 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:54.682898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-bac902de-5462-4a39-be80-f588a9903681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:54.751023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 513/1022] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:00:54.773387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03f3f0-86c5-41cb-90e1-022fa33e5194 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:54.799581 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:54.882871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03f3f0-86c5-41cb-90e1-022fa33e5194 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:54.917613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836701a-5e1d-4dec-a7d7-dc117c2d1a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:54.918433 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836701a-5e1d-4dec-a7d7-dc117c2d1a12 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:54.918818 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836701a-5e1d-4dec-a7d7-dc117c2d1a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:54.919006 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836701a-5e1d-4dec-a7d7-dc117c2d1a12 None admin] PATH_INFO: `/v3/services/image` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:54.919665 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836701a-5e1d-4dec-a7d7-dc117c2d1a12 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:54.921037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836701a-5e1d-4dec-a7d7-dc117c2d1a12 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-c836701a-5e1d-4dec-a7d7-dc117c2d1a12 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:54.932651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:54.934181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:54.934181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:54.934181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:54.934181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:54.934181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:54.934181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:54.934181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:54.934181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:54.934181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:54.934181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:54.934181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:54.934181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:54.934181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:54.934181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:54.934181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:54.934181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:54.935200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:54.935200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:54.935200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:54.935200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:54.935200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:54.935200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:54.935200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:54.935200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:54.935200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:54.935200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:54.935200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:54.935200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:54.935200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:00:54.935200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:54.935200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 510/1023] 10.222.0.44 () {56 vars in 1105 bytes} [Wed Jul 20 09:00:54 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:54.941334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801aa8f-f8b0-4491-a374-b09799c53cd2 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:54.941334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801aa8f-f8b0-4491-a374-b09799c53cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:54.941334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801aa8f-f8b0-4491-a374-b09799c53cd2 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:54.941334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 514/1024] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:54.941582 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836701a-5e1d-4dec-a7d7-dc117c2d1a12 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:54.944894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fc3e0-275c-4daa-a90a-1692d6bcf46b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:54.945188 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fc3e0-275c-4daa-a90a-1692d6bcf46b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:54.945473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fc3e0-275c-4daa-a90a-1692d6bcf46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:54.962294 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-362fc3e0-275c-4daa-a90a-1692d6bcf46b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:54.963521 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-362fc3e0-275c-4daa-a90a-1692d6bcf46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:55.035207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:55.035741 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 515/1025] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:00:55.058348 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836701a-5e1d-4dec-a7d7-dc117c2d1a12 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:55.063905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fc3e0-275c-4daa-a90a-1692d6bcf46b None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:55.098954 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ad381-6987-47c5-9572-27ded2f7c2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:55.099623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ad381-6987-47c5-9572-27ded2f7c2bd None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:55.099913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ad381-6987-47c5-9572-27ded2f7c2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:55.100194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ad381-6987-47c5-9572-27ded2f7c2bd None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:55.100905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ad381-6987-47c5-9572-27ded2f7c2bd None admin] RBAC: Adding query filter params (name=image) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:55.101215 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ad381-6987-47c5-9572-27ded2f7c2bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:55.102930 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ad381-6987-47c5-9572-27ded2f7c2bd None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:55.112823 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 511/1026] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:55.113818 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ad381-6987-47c5-9572-27ded2f7c2bd None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:55.135657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:55.183595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fc3e0-275c-4daa-a90a-1692d6bcf46b None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:55.220312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36844821-be86-49b1-ae87-15ca731c6ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:55.221129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36844821-be86-49b1-ae87-15ca731c6ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:55.221387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ad381-6987-47c5-9572-27ded2f7c2bd None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:55.221495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36844821-be86-49b1-ae87-15ca731c6ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:55.221685 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36844821-be86-49b1-ae87-15ca731c6ab4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:55.232184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36844821-be86-49b1-ae87-15ca731c6ab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:55.240280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-36844821-be86-49b1-ae87-15ca731c6ab4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:00:55.240280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:55.240280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:55.240280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:55.240280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:55.240280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:55.240280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:55.240280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:55.240280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:55.240280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:55.240280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:55.240280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:55.240280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:55.240280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:55.240280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:55.240280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:55.241223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:55.241223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:55.241223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:55.241223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:55.241223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:55.241223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:55.241223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:55.241223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:55.241223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:55.241223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:55.241223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:55.241223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:55.241223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:55.241223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:55.241223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:55.241223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:55.242100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:55.242100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:55.242100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:55.242100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:55.242100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:55.242100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:55.242100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:55.242100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:55.242100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:55.242100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:55.242100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:00:55.242100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:55.246613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 516/1027] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Jul 20 09:00:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:55.247514 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36844821-be86-49b1-ae87-15ca731c6ab4 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:55.258731 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d40175-e49d-4895-8ef5-61c859cc35e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:55.259422 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d40175-e49d-4895-8ef5-61c859cc35e2 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:55.259708 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d40175-e49d-4895-8ef5-61c859cc35e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:55.259990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d40175-e49d-4895-8ef5-61c859cc35e2 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:55.260707 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d40175-e49d-4895-8ef5-61c859cc35e2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:55.261016 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d40175-e49d-4895-8ef5-61c859cc35e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:55.263060 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d40175-e49d-4895-8ef5-61c859cc35e2 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:55.270721 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 512/1028] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:55.275200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d40175-e49d-4895-8ef5-61c859cc35e2 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:55.291881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:55.361985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36844821-be86-49b1-ae87-15ca731c6ab4 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:55.379010 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:55.383887 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d40175-e49d-4895-8ef5-61c859cc35e2 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:55.398634 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed1443-1d73-4a97-8c66-a45d3e0a3431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:55.399260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed1443-1d73-4a97-8c66-a45d3e0a3431 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:55.399540 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed1443-1d73-4a97-8c66-a45d3e0a3431 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:55.399813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed1443-1d73-4a97-8c66-a45d3e0a3431 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:55.400562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed1443-1d73-4a97-8c66-a45d3e0a3431 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:55.400871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed1443-1d73-4a97-8c66-a45d3e0a3431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:55.402218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed1443-1d73-4a97-8c66-a45d3e0a3431 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:55.411081 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 517/1029] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Jul 20 09:00:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:55.416401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed1443-1d73-4a97-8c66-a45d3e0a3431 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:55.420214 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6ea43-07f2-485c-b4f4-2039ef77fcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:55.420926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6ea43-07f2-485c-b4f4-2039ef77fcdf None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:55.421210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6ea43-07f2-485c-b4f4-2039ef77fcdf None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:55.421489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6ea43-07f2-485c-b4f4-2039ef77fcdf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:55.422276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a6ea43-07f2-485c-b4f4-2039ef77fcdf None admin] RBAC: Adding query filter params (interface=public, service_id=8b53592bea96421fb8373ffdb1a1efb7, region_id=RegionOne) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:55.422586 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a6ea43-07f2-485c-b4f4-2039ef77fcdf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:55.423983 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a6ea43-07f2-485c-b4f4-2039ef77fcdf None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:55.433600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 513/1030] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Jul 20 09:00:55 2022] GET /identity/v3/endpoints?service_id=8b53592bea96421fb8373ffdb1a1efb7&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:55.435899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6ea43-07f2-485c-b4f4-2039ef77fcdf None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:55.528109 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:55.531850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed1443-1d73-4a97-8c66-a45d3e0a3431 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:55.557367 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6ea43-07f2-485c-b4f4-2039ef77fcdf None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:55.573622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6dcf66-21c5-4618-be29-d5804f2c7f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:55.575435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dcf66-21c5-4618-be29-d5804f2c7f7a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:55.576202 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dcf66-21c5-4618-be29-d5804f2c7f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:55.576696 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dcf66-21c5-4618-be29-d5804f2c7f7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:55.591349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6dcf66-21c5-4618-be29-d5804f2c7f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:55.592485 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbe546-38dc-41e8-82c7-218e92abab05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:55.593145 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbe546-38dc-41e8-82c7-218e92abab05 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:55.593425 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbe546-38dc-41e8-82c7-218e92abab05 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:55.593698 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbe546-38dc-41e8-82c7-218e92abab05 None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:55.593966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6dcf66-21c5-4618-be29-d5804f2c7f7a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:55.594378 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbe546-38dc-41e8-82c7-218e92abab05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:55.595759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbe546-38dc-41e8-82c7-218e92abab05 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:55.610704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 514/1031] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jul 20 09:00:55 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:00:55.612259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-ab6dcf66-21c5-4618-be29-d5804f2c7f7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:55.612259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:55.612259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:55.612259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:55.612259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:55.612259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:55.612259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:55.612259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:55.612259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:55.612259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:55.612259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:55.612259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:55.612259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:55.612259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:55.612259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:55.613150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:55.614062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:55.614062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:55.614062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:55.614062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:55.614062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 518/1032] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:55.619648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbe546-38dc-41e8-82c7-218e92abab05 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:55.731443 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbe546-38dc-41e8-82c7-218e92abab05 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:55.763934 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a3060-39fe-43c9-950d-102d49450c13 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:55.764476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a3060-39fe-43c9-950d-102d49450c13 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:55.764568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a3060-39fe-43c9-950d-102d49450c13 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:55.765889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 519/1033] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:55.769153 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec0466-f412-475d-adf0-6ca2dea4b85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:55.769875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec0466-f412-475d-adf0-6ca2dea4b85f None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:55.770165 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec0466-f412-475d-adf0-6ca2dea4b85f None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:55.770448 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec0466-f412-475d-adf0-6ca2dea4b85f None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:55.771031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457bc58-ab4a-47ce-b525-d8ce6ea772f2 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:55.771227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec0466-f412-475d-adf0-6ca2dea4b85f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:55.771342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457bc58-ab4a-47ce-b525-d8ce6ea772f2 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:55.771531 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec0466-f412-475d-adf0-6ca2dea4b85f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:55.771634 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e457bc58-ab4a-47ce-b525-d8ce6ea772f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:55.772911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec0466-f412-475d-adf0-6ca2dea4b85f None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:55.786439 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 515/1034] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:55.791908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec0466-f412-475d-adf0-6ca2dea4b85f None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:55.792045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-e457bc58-ab4a-47ce-b525-d8ce6ea772f2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:55.793169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e457bc58-ab4a-47ce-b525-d8ce6ea772f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:55.794852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:55.819628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:55.872996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 520/1035] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:00:55.895957 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec0466-f412-475d-adf0-6ca2dea4b85f None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:55.897131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457bc58-ab4a-47ce-b525-d8ce6ea772f2 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:55.932232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0e87f-e0c9-4099-b886-1f7a2f2a6a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:55.933025 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0e87f-e0c9-4099-b886-1f7a2f2a6a26 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:55.933304 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0e87f-e0c9-4099-b886-1f7a2f2a6a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:55.933601 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0e87f-e0c9-4099-b886-1f7a2f2a6a26 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:55.941096 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0e87f-e0c9-4099-b886-1f7a2f2a6a26 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:55.942542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0e87f-e0c9-4099-b886-1f7a2f2a6a26 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:55.958456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-85f0e87f-e0c9-4099-b886-1f7a2f2a6a26 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:00:55.958456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:55.958456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:55.958456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:55.958456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:55.958456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:55.958456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:55.958456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:55.958456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:55.958456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:55.958456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:55.958456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:55.958456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:00:55.958456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:00:55.958456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:00:55.958456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:55.959609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:55.960576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:55.960576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:55.960576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:00:55.960576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:00:55.960576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:00:55.960576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:00:55.960576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:00:55.960576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:00:55.960576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:00:55.960576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:55.960576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 516/1036] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jul 20 09:00:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:55.965228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0e87f-e0c9-4099-b886-1f7a2f2a6a26 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:56.008750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e457bc58-ab4a-47ce-b525-d8ce6ea772f2 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:56.031102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:56.047045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b46fa-946b-4d7c-9306-b53520bce724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:56.047844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b46fa-946b-4d7c-9306-b53520bce724 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:56.048128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b46fa-946b-4d7c-9306-b53520bce724 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:56.048414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b46fa-946b-4d7c-9306-b53520bce724 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:56.049586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b46fa-946b-4d7c-9306-b53520bce724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:56.051059 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b46fa-946b-4d7c-9306-b53520bce724 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:56.052959 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 521/1037] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:56.058339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b46fa-946b-4d7c-9306-b53520bce724 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:56.078247 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0e87f-e0c9-4099-b886-1f7a2f2a6a26 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:56.094716 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:56.114880 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca97477-8da5-474b-a211-30655be59159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:56.115554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca97477-8da5-474b-a211-30655be59159 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:56.115836 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca97477-8da5-474b-a211-30655be59159 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:56.116111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca97477-8da5-474b-a211-30655be59159 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:56.116939 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca97477-8da5-474b-a211-30655be59159 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:56.117244 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca97477-8da5-474b-a211-30655be59159 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:56.118661 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca97477-8da5-474b-a211-30655be59159 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:56.142831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 517/1038] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jul 20 09:00:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:56.145914 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca97477-8da5-474b-a211-30655be59159 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:56.169260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b46fa-946b-4d7c-9306-b53520bce724 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:56.205591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c84f5-06a8-4497-a0a3-52b57d62b44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:56.206260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c84f5-06a8-4497-a0a3-52b57d62b44e None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:56.206542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c84f5-06a8-4497-a0a3-52b57d62b44e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:56.206853 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c84f5-06a8-4497-a0a3-52b57d62b44e None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:56.207562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c84f5-06a8-4497-a0a3-52b57d62b44e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:56.208965 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c84f5-06a8-4497-a0a3-52b57d62b44e None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:56.231958 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 522/1039] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 09:00:56 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:00:56.253981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca97477-8da5-474b-a211-30655be59159 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:56.279360 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:56.296642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b0d84-aaaa-416b-8209-72d3caedc309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:56.297359 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b0d84-aaaa-416b-8209-72d3caedc309 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:56.297645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b0d84-aaaa-416b-8209-72d3caedc309 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:56.297983 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b0d84-aaaa-416b-8209-72d3caedc309 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:56.373258 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b0d84-aaaa-416b-8209-72d3caedc309 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:56.374761 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b0d84-aaaa-416b-8209-72d3caedc309 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:56.486588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 518/1040] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:00:56 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 09:00:56.492220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c84f5-06a8-4497-a0a3-52b57d62b44e None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:56.584502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:56.602148 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c84f5-06a8-4497-a0a3-52b57d62b44e None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:56.636422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5c5bc-85d5-4dc6-993b-96d22f321ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:56.637739 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5c5bc-85d5-4dc6-993b-96d22f321ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:56.638018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5c5bc-85d5-4dc6-993b-96d22f321ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:56.638288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5c5bc-85d5-4dc6-993b-96d22f321ec3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:56.648893 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5c5bc-85d5-4dc6-993b-96d22f321ec3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:56.650333 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5c5bc-85d5-4dc6-993b-96d22f321ec3 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:56.667456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-19a5c5bc-85d5-4dc6-993b-96d22f321ec3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:56.667456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:56.667456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:56.667456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:56.667456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:56.667456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:56.667456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:56.667456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:56.667456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:56.667456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:56.667456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:56.667456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:56.667456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:56.667456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:56.667456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:56.669004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:56.669911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:56.669911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:56.669911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:56.669911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:56.669911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 523/1041] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:56.676081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b0d84-aaaa-416b-8209-72d3caedc309 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:56.698434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:56.782928 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b0d84-aaaa-416b-8209-72d3caedc309 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:56.817859 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416890a0-d35f-478c-8f81-ff8861a0a6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:56.818440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416890a0-d35f-478c-8f81-ff8861a0a6dd None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:56.818743 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416890a0-d35f-478c-8f81-ff8861a0a6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:56.819022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416890a0-d35f-478c-8f81-ff8861a0a6dd None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:56.820000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416890a0-d35f-478c-8f81-ff8861a0a6dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:56.820303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416890a0-d35f-478c-8f81-ff8861a0a6dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:56.821656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416890a0-d35f-478c-8f81-ff8861a0a6dd None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:56.837487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 519/1042] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:56.838718 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5c5bc-85d5-4dc6-993b-96d22f321ec3 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:56.940372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5c5bc-85d5-4dc6-993b-96d22f321ec3 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:56.975222 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c988c3-a771-4aa8-a94b-c58056be44cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:56.976063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c988c3-a771-4aa8-a94b-c58056be44cb None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:56.976354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c988c3-a771-4aa8-a94b-c58056be44cb None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:56.976632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c988c3-a771-4aa8-a94b-c58056be44cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:56.979311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:56.985757 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c988c3-a771-4aa8-a94b-c58056be44cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:56.987250 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c988c3-a771-4aa8-a94b-c58056be44cb None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:57.001611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-c3c988c3-a771-4aa8-a94b-c58056be44cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:57.001611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:57.001611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:57.001611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:57.001611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:57.001611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:57.001611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:57.001611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:57.001611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:57.001611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:57.001611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:57.001611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:57.001611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:00:57.001611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:00:57.001611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:57.002510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:57.008488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:00:57.008488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:00:57.008488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:00:57.008488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:00:57.008488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:00:57.008488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:00:57.008488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:00:57.008488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:57.008488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 524/1043] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:00:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:57.010334 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416890a0-d35f-478c-8f81-ff8861a0a6dd None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:57.095991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:57.116025 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416890a0-d35f-478c-8f81-ff8861a0a6dd None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:57.149868 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389fede-0ec2-4d65-806b-ee74378d4a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:57.150525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389fede-0ec2-4d65-806b-ee74378d4a77 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:57.150858 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389fede-0ec2-4d65-806b-ee74378d4a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:57.151137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389fede-0ec2-4d65-806b-ee74378d4a77 None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:57.151915 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389fede-0ec2-4d65-806b-ee74378d4a77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:57.152223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389fede-0ec2-4d65-806b-ee74378d4a77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:57.153845 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389fede-0ec2-4d65-806b-ee74378d4a77 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:57.166709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 520/1044] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:00:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:57.172528 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c988c3-a771-4aa8-a94b-c58056be44cb None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:57.274541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c988c3-a771-4aa8-a94b-c58056be44cb None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:57.309768 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662f0da-ab65-48e5-a241-4e1d5f4bae8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:57.310392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662f0da-ab65-48e5-a241-4e1d5f4bae8a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:57.310686 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662f0da-ab65-48e5-a241-4e1d5f4bae8a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:57.310978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662f0da-ab65-48e5-a241-4e1d5f4bae8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:57.311815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662f0da-ab65-48e5-a241-4e1d5f4bae8a None admin] RBAC: Adding query filter params (role.id=a2636fed60634c3997e3c9213a818395, scope.project.id=a498284274454d25b1b118a88996e522, user.id=be08c43f0d0c4a8194596e54500b44ed) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:57.312129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662f0da-ab65-48e5-a241-4e1d5f4bae8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:57.313555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662f0da-ab65-48e5-a241-4e1d5f4bae8a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:57.326735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 525/1045] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:00:57 2022] GET /identity/v3/role_assignments?user.id=be08c43f0d0c4a8194596e54500b44ed&scope.project.id=a498284274454d25b1b118a88996e522&role.id=a2636fed60634c3997e3c9213a818395 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:57.692872 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932e9ce-c67d-49af-8e9a-a5ba0cbaf8d3 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:57.693219 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932e9ce-c67d-49af-8e9a-a5ba0cbaf8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:57.693433 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932e9ce-c67d-49af-8e9a-a5ba0cbaf8d3 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:57.695116 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 521/1046] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:57.700352 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705d832-1205-461d-82ac-ba16f11c1436 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:57.700713 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705d832-1205-461d-82ac-ba16f11c1436 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:57.701091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705d832-1205-461d-82ac-ba16f11c1436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:57.718354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-0705d832-1205-461d-82ac-ba16f11c1436 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:57.719573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-0705d832-1205-461d-82ac-ba16f11c1436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:57.776272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:57.788779 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 526/1047] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:00:57.812506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932e9ce-c67d-49af-8e9a-a5ba0cbaf8d3 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:57.916563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932e9ce-c67d-49af-8e9a-a5ba0cbaf8d3 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:57.952353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309b0d3-0eb3-4d56-879b-5b4a7df2c6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:57.953140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309b0d3-0eb3-4d56-879b-5b4a7df2c6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:57.953426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309b0d3-0eb3-4d56-879b-5b4a7df2c6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:57.953704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309b0d3-0eb3-4d56-879b-5b4a7df2c6d0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:57.954361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309b0d3-0eb3-4d56-879b-5b4a7df2c6d0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:57.955766 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309b0d3-0eb3-4d56-879b-5b4a7df2c6d0 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-2309b0d3-0eb3-4d56-879b-5b4a7df2c6d0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:57.968048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:57.969102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:57.969102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:57.969102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:57.969102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:57.969102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:57.969102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:57.969102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:00:57.969102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:00:57.969102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:00:57.969102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:00:57.969102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:57.969102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:57.969102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:57.969102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:57.969102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:57.969102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:57.969959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:57.969959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:57.969959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:57.969959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:57.969959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:57.969959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:57.969959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:57.969959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:57.969959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:57.969959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:57.969959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:00:57.969959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:00:57.969959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 09:00:57.969959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:57.971696 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 522/1048] 10.222.0.44 () {56 vars in 1105 bytes} [Wed Jul 20 09:00:57 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:57.975238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705d832-1205-461d-82ac-ba16f11c1436 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:58.076444 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705d832-1205-461d-82ac-ba16f11c1436 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:58.110905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37979ce-ea26-4621-9a31-fb60eb192f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:58.111557 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37979ce-ea26-4621-9a31-fb60eb192f27 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:58.111842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37979ce-ea26-4621-9a31-fb60eb192f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:58.112122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37979ce-ea26-4621-9a31-fb60eb192f27 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:58.112830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37979ce-ea26-4621-9a31-fb60eb192f27 None admin] RBAC: Adding query filter params (name=image) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:58.113141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37979ce-ea26-4621-9a31-fb60eb192f27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:58.114510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37979ce-ea26-4621-9a31-fb60eb192f27 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:58.123965 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 527/1049] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:58.125463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309b0d3-0eb3-4d56-879b-5b4a7df2c6d0 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:58.129351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:58.228544 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309b0d3-0eb3-4d56-879b-5b4a7df2c6d0 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:58.262581 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc07ee1-94cb-4802-a950-730322c3e05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:58.263272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc07ee1-94cb-4802-a950-730322c3e05c None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:58.263560 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc07ee1-94cb-4802-a950-730322c3e05c None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:58.263841 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc07ee1-94cb-4802-a950-730322c3e05c None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:58.264549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc07ee1-94cb-4802-a950-730322c3e05c None admin] RBAC: Adding query filter params (type=image) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:58.264866 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc07ee1-94cb-4802-a950-730322c3e05c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:58.266281 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc07ee1-94cb-4802-a950-730322c3e05c None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:58.276436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 523/1050] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:58.278043 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37979ce-ea26-4621-9a31-fb60eb192f27 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:58.334715 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:58.361425 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74d702-65b2-4cf2-bb2e-f17fc0190677 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:58.361724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74d702-65b2-4cf2-bb2e-f17fc0190677 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:58.362006 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74d702-65b2-4cf2-bb2e-f17fc0190677 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:58.363663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 524/1051] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:00:58.368346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98380929-f321-4897-80f5-268e007d9f40 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:58.368719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98380929-f321-4897-80f5-268e007d9f40 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:58.368974 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98380929-f321-4897-80f5-268e007d9f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:58.382332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37979ce-ea26-4621-9a31-fb60eb192f27 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:58.386102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-98380929-f321-4897-80f5-268e007d9f40 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:58.387372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-98380929-f321-4897-80f5-268e007d9f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:58.419972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123803c-6e55-40ee-bbe9-1f5ad200010a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:58.420658 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123803c-6e55-40ee-bbe9-1f5ad200010a None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:58.420943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123803c-6e55-40ee-bbe9-1f5ad200010a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:58.421220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123803c-6e55-40ee-bbe9-1f5ad200010a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:58.421982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123803c-6e55-40ee-bbe9-1f5ad200010a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:58.423376 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123803c-6e55-40ee-bbe9-1f5ad200010a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:58.446153 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 528/1052] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jul 20 09:00:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:00:58.460547 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:58.470543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 525/1053] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:00:58.494572 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123803c-6e55-40ee-bbe9-1f5ad200010a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:58.611303 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123803c-6e55-40ee-bbe9-1f5ad200010a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:58.622568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:58.646935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a358e13-f4d2-4d36-a6ba-82e69b8ecd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:58.647740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a358e13-f4d2-4d36-a6ba-82e69b8ecd2c None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:58.648028 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a358e13-f4d2-4d36-a6ba-82e69b8ecd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:58.648308 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a358e13-f4d2-4d36-a6ba-82e69b8ecd2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:58.659434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a358e13-f4d2-4d36-a6ba-82e69b8ecd2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:58.667157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-4a358e13-f4d2-4d36-a6ba-82e69b8ecd2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:00:58.667157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:58.667157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:58.667157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:58.667157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:58.667157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:58.667157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:58.667157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:58.667157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:58.667157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:58.667157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:58.667157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:58.667157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:58.667157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:58.667157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:58.667157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:58.668193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:58.668193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:58.668193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:58.668193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:58.668193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:58.668193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:58.668193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:58.668193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:58.668193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:58.668193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:58.668193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:58.668193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:58.668193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:58.668193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:58.668193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:58.668193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:58.669134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:58.669134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:58.669134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:58.669134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:58.669134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:58.669134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:58.669134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:58.669134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:58.669134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:58.669134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:58.669134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:00:58.669134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:58.669134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 529/1054] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:00:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:58.674665 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98380929-f321-4897-80f5-268e007d9f40 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:58.779450 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98380929-f321-4897-80f5-268e007d9f40 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:58.815837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cdfe99-41bb-4bb8-b057-4257658b869b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:58.816570 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cdfe99-41bb-4bb8-b057-4257658b869b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:58.816786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cdfe99-41bb-4bb8-b057-4257658b869b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:58.817067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cdfe99-41bb-4bb8-b057-4257658b869b None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:58.817822 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cdfe99-41bb-4bb8-b057-4257658b869b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:58.818134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cdfe99-41bb-4bb8-b057-4257658b869b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:58.819798 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cdfe99-41bb-4bb8-b057-4257658b869b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:58.829191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 526/1055] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:00:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:58.835192 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a358e13-f4d2-4d36-a6ba-82e69b8ecd2c None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:58.939950 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a358e13-f4d2-4d36-a6ba-82e69b8ecd2c None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:58.975182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1d249-913c-44c1-8398-d5e6da48822e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:58.976056 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1d249-913c-44c1-8398-d5e6da48822e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:58.976609 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1d249-913c-44c1-8398-d5e6da48822e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:58.976609 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1d249-913c-44c1-8398-d5e6da48822e None admin] PATH_INFO: `/v3/users/a405801a2c6140949d5725ebb8623700` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:58.990444 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1d249-913c-44c1-8398-d5e6da48822e None admin] RBAC: Authorizing `identity:get_user(user_id=a405801a2c6140949d5725ebb8623700)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:58.991923 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1d249-913c-44c1-8398-d5e6da48822e None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:58.998219 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 530/1056] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 09:00:58 2022] GET /identity/v3/users/a405801a2c6140949d5725ebb8623700 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:58.999497 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cdfe99-41bb-4bb8-b057-4257658b869b None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:59.059499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:59.106750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cdfe99-41bb-4bb8-b057-4257658b869b None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:59.141461 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f230b8-9de1-453a-973d-70c06a0b3a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:59.142122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f230b8-9de1-453a-973d-70c06a0b3a19 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:59.142410 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f230b8-9de1-453a-973d-70c06a0b3a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:59.142699 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f230b8-9de1-453a-973d-70c06a0b3a19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:59.211027 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f230b8-9de1-453a-973d-70c06a0b3a19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:59.212402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f230b8-9de1-453a-973d-70c06a0b3a19 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:59.269686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:59.320685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 527/1057] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:00:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:00:59.321955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1d249-913c-44c1-8398-d5e6da48822e None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:59.344723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62034046-6424-49bc-9791-fde4d9043097 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:59.345371 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62034046-6424-49bc-9791-fde4d9043097 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:59.345371 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62034046-6424-49bc-9791-fde4d9043097 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:59.346650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 528/1058] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:00:59.351349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d05aa-d2ea-4350-9615-ce2d4896dbcf None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:59.351633 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d05aa-d2ea-4350-9615-ce2d4896dbcf None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:59.351919 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d05aa-d2ea-4350-9615-ce2d4896dbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:59.352354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:59.368507 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-e27d05aa-d2ea-4350-9615-ce2d4896dbcf None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:00:59.369689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e27d05aa-d2ea-4350-9615-ce2d4896dbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:00:59.439846 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 529/1059] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:00:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:00:59.440273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1d249-913c-44c1-8398-d5e6da48822e None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:59.463401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d05aa-d2ea-4350-9615-ce2d4896dbcf None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:59.483385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1f0d6-3348-434e-9110-b2cbab9279fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:59.484426 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1f0d6-3348-434e-9110-b2cbab9279fe None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:59.484718 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1f0d6-3348-434e-9110-b2cbab9279fe None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:59.485002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1f0d6-3348-434e-9110-b2cbab9279fe None admin] PATH_INFO: `/v3/projects/e112dfa169e04071b6f410eca228826e` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:59.486305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1f0d6-3348-434e-9110-b2cbab9279fe None admin] RBAC: Authorizing `identity:get_project(project_id=e112dfa169e04071b6f410eca228826e)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:59.487781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1f0d6-3348-434e-9110-b2cbab9279fe None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:59.490005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 531/1060] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 09:00:59 2022] GET /identity/v3/projects/e112dfa169e04071b6f410eca228826e => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:59.495479 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1f0d6-3348-434e-9110-b2cbab9279fe None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:59.564746 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:59.583004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d05aa-d2ea-4350-9615-ce2d4896dbcf None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:59.604711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:00:59.612687 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1f0d6-3348-434e-9110-b2cbab9279fe None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:59.623122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c444cfb-3ef2-47f8-adac-30199a9fe546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:59.623934 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c444cfb-3ef2-47f8-adac-30199a9fe546 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:59.624218 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c444cfb-3ef2-47f8-adac-30199a9fe546 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:59.624494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c444cfb-3ef2-47f8-adac-30199a9fe546 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:59.635095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c444cfb-3ef2-47f8-adac-30199a9fe546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:00:59.643389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-3c444cfb-3ef2-47f8-adac-30199a9fe546 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:00:59.643389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:59.643389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:59.643389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:59.643389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:59.643389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:59.643389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:59.643389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:59.643389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:59.643389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:59.643389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:59.643389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:59.643389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:00:59.643389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:00:59.643389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:00:59.643389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:00:59.644333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:00:59.644333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:00:59.644333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:00:59.644333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:00:59.644333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:59.644333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:59.644333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:59.644333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:59.644333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:59.644333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:59.644333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:59.644333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:59.644333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:59.644333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:59.644333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:59.644333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:59.645177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:59.645177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:59.645177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:59.645177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:59.645177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:00:59.645177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:00:59.645177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:00:59.645177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:00:59.645177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:00:59.645177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:00:59.645177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:00:59.645177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:00:59.645177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 530/1061] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Jul 20 09:00:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:00:59.650395 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c444cfb-3ef2-47f8-adac-30199a9fe546 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:59.653314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26749e68-6151-4d07-b13f-13cabb3c74b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:59.653985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26749e68-6151-4d07-b13f-13cabb3c74b7 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:59.654273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26749e68-6151-4d07-b13f-13cabb3c74b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:59.654555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26749e68-6151-4d07-b13f-13cabb3c74b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:59.655443 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26749e68-6151-4d07-b13f-13cabb3c74b7 None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=e112dfa169e04071b6f410eca228826e, user.id=a405801a2c6140949d5725ebb8623700) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:59.655762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26749e68-6151-4d07-b13f-13cabb3c74b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:59.657191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26749e68-6151-4d07-b13f-13cabb3c74b7 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:59.674713 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 532/1062] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:00:59 2022] GET /identity/v3/role_assignments?user.id=a405801a2c6140949d5725ebb8623700&scope.project.id=e112dfa169e04071b6f410eca228826e&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:00:59.768325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c444cfb-3ef2-47f8-adac-30199a9fe546 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:59.804649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f2ea7-2af8-4662-9f84-538bc18627d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:59.806651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f2ea7-2af8-4662-9f84-538bc18627d7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:59.806651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f2ea7-2af8-4662-9f84-538bc18627d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:59.806651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f2ea7-2af8-4662-9f84-538bc18627d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:59.806651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f2ea7-2af8-4662-9f84-538bc18627d7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:00:59.806981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f2ea7-2af8-4662-9f84-538bc18627d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:59.808994 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f2ea7-2af8-4662-9f84-538bc18627d7 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:59.817200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 531/1063] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Jul 20 09:00:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:00:59.823064 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26749e68-6151-4d07-b13f-13cabb3c74b7 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:00:59.924693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26749e68-6151-4d07-b13f-13cabb3c74b7 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:00:59.960216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235204e0-83e9-408b-b6a8-897cfa361137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:00:59.961012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235204e0-83e9-408b-b6a8-897cfa361137 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:00:59.961293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235204e0-83e9-408b-b6a8-897cfa361137 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:00:59.961569 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235204e0-83e9-408b-b6a8-897cfa361137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:00:59.971995 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235204e0-83e9-408b-b6a8-897cfa361137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:00:59.973482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235204e0-83e9-408b-b6a8-897cfa361137 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:00:59.991077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-235204e0-83e9-408b-b6a8-897cfa361137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:59.991077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:00:59.991077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:00:59.991077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:00:59.991077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:00:59.991077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:00:59.991077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:00:59.991077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:00:59.991077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:00:59.991077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:00:59.991077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:00:59.991077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:00:59.991077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:00:59.991077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:00:59.991077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:00:59.992429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:00:59.993317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:00:59.993317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:00:59.993317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:00:59.993317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:00:59.994769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 533/1064] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:00:59.999282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f2ea7-2af8-4662-9f84-538bc18627d7 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:00.010745 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:00.103503 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f2ea7-2af8-4662-9f84-538bc18627d7 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:00.138104 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783393b0-2d04-4ec4-a8a4-29867bd7f837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:00.138806 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783393b0-2d04-4ec4-a8a4-29867bd7f837 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:00.139098 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783393b0-2d04-4ec4-a8a4-29867bd7f837 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:00.139383 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783393b0-2d04-4ec4-a8a4-29867bd7f837 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:00.140109 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783393b0-2d04-4ec4-a8a4-29867bd7f837 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:00.140423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783393b0-2d04-4ec4-a8a4-29867bd7f837 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:00.141809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783393b0-2d04-4ec4-a8a4-29867bd7f837 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:00.154458 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 532/1065] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:00.159968 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235204e0-83e9-408b-b6a8-897cfa361137 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:00.175807 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:00.264174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235204e0-83e9-408b-b6a8-897cfa361137 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:00.299322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8d90e-8fc0-44a8-a703-ed5b3a1cb747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:00.300119 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8d90e-8fc0-44a8-a703-ed5b3a1cb747 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:00.300408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8d90e-8fc0-44a8-a703-ed5b3a1cb747 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:00.300689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8d90e-8fc0-44a8-a703-ed5b3a1cb747 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:00.307806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8d90e-8fc0-44a8-a703-ed5b3a1cb747 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:00.309965 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8d90e-8fc0-44a8-a703-ed5b3a1cb747 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:00.324506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-eec8d90e-8fc0-44a8-a703-ed5b3a1cb747 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:01:00.324506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:00.324506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:00.324506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:00.324506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:00.324506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:00.324506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:00.324506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:00.324506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:00.324506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:00.324506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:00.324506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:00.324506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:01:00.324506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:01:00.324506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:01:00.324506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:00.325435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:00.326427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:00.326427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:00.326427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:01:00.326427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:01:00.326427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:01:00.326427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:01:00.326427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:01:00.326427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:01:00.326427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:01:00.326427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:01:00.330705 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 534/1066] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jul 20 09:01:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:01:00.331455 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783393b0-2d04-4ec4-a8a4-29867bd7f837 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:00.400867 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:00.438970 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783393b0-2d04-4ec4-a8a4-29867bd7f837 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:00.475542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c2665e-7f53-4dd1-9207-fcec377bd314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:00.476213 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2665e-7f53-4dd1-9207-fcec377bd314 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:00.476500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2665e-7f53-4dd1-9207-fcec377bd314 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:00.476779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2665e-7f53-4dd1-9207-fcec377bd314 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:00.477611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c2665e-7f53-4dd1-9207-fcec377bd314 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:00.477922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c2665e-7f53-4dd1-9207-fcec377bd314 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:00.479396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c2665e-7f53-4dd1-9207-fcec377bd314 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:00.502721 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 533/1067] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jul 20 09:01:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:00.504737 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8d90e-8fc0-44a8-a703-ed5b3a1cb747 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:00.510804 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944192-80b9-47cb-b04d-83161bb27e5a None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:00.511312 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944192-80b9-47cb-b04d-83161bb27e5a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:00.511792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944192-80b9-47cb-b04d-83161bb27e5a None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:00.514583 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 534/1068] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:01:00.519317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d06fe-1860-49af-8611-c521d18f688b None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:00.519818 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d06fe-1860-49af-8611-c521d18f688b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:00.520321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d06fe-1860-49af-8611-c521d18f688b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:00.538333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-cc3d06fe-1860-49af-8611-c521d18f688b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:00.539825 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-cc3d06fe-1860-49af-8611-c521d18f688b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:00.582163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:00.615261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8d90e-8fc0-44a8-a703-ed5b3a1cb747 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:00.615446 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 535/1069] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:01:00.638331 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d06fe-1860-49af-8611-c521d18f688b None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:00.655142 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416197f5-1ca8-45bb-a75a-444064627db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:00.655871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416197f5-1ca8-45bb-a75a-444064627db7 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:00.656159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416197f5-1ca8-45bb-a75a-444064627db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:00.656438 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416197f5-1ca8-45bb-a75a-444064627db7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:00.666516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416197f5-1ca8-45bb-a75a-444064627db7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:00.668696 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416197f5-1ca8-45bb-a75a-444064627db7 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:00.686760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-416197f5-1ca8-45bb-a75a-444064627db7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:01:00.686760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:00.686760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:00.686760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:00.686760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:00.686760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:00.686760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:00.686760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:00.686760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:00.686760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:00.686760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:00.686760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:00.686760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:01:00.686760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:01:00.686760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:00.687673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:00.688717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:01:00.688717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:01:00.688717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:01:00.688717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:01:00.688717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 535/1070] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:01:00.693497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416197f5-1ca8-45bb-a75a-444064627db7 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:00.698501 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:00.743598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d06fe-1860-49af-8611-c521d18f688b None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:00.781556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26c255-cbd0-4e89-8374-a72272b1e5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:00.782237 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26c255-cbd0-4e89-8374-a72272b1e5fa None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:00.782525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26c255-cbd0-4e89-8374-a72272b1e5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:00.782861 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26c255-cbd0-4e89-8374-a72272b1e5fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:00.809557 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416197f5-1ca8-45bb-a75a-444064627db7 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:00.834737 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:00.847848 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449023da-edad-4f73-9792-ca7ec8324b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:00.848766 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449023da-edad-4f73-9792-ca7ec8324b74 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:00.849229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449023da-edad-4f73-9792-ca7ec8324b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:00.849798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449023da-edad-4f73-9792-ca7ec8324b74 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:00.850886 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449023da-edad-4f73-9792-ca7ec8324b74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:00.851374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449023da-edad-4f73-9792-ca7ec8324b74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:00.852664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26c255-cbd0-4e89-8374-a72272b1e5fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:00.853448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449023da-edad-4f73-9792-ca7ec8324b74 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:00.854145 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26c255-cbd0-4e89-8374-a72272b1e5fa None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:00.866866 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 536/1071] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:01:00.872367 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449023da-edad-4f73-9792-ca7ec8324b74 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:00.917294 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:00.970330 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 536/1072] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:01:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 09:01:00.975261 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26c255-cbd0-4e89-8374-a72272b1e5fa None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:00.984250 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449023da-edad-4f73-9792-ca7ec8324b74 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:01.021249 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a332ae-7399-4f72-9144-6fe4ce0776dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:01.022294 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a332ae-7399-4f72-9144-6fe4ce0776dd None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:01.022576 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a332ae-7399-4f72-9144-6fe4ce0776dd None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:01.023029 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a332ae-7399-4f72-9144-6fe4ce0776dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:01.030953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a332ae-7399-4f72-9144-6fe4ce0776dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:01.032372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a332ae-7399-4f72-9144-6fe4ce0776dd None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:01.046881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-85a332ae-7399-4f72-9144-6fe4ce0776dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:01:01.046881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:01.046881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:01.046881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:01.046881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:01.046881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:01.046881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:01.046881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:01.046881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:01.046881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:01.046881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:01.046881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:01.046881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:01:01.046881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:01:01.046881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:01.047809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:01.054039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:01:01.054039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:01:01.054039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:01:01.054039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:01:01.054039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:01:01.054039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:01:01.054039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:01:01.054039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:01:01.054039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 537/1073] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:01:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:01:01.054039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a332ae-7399-4f72-9144-6fe4ce0776dd None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:01.081110 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26c255-cbd0-4e89-8374-a72272b1e5fa None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:01.101071 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:01.118919 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc809e-a178-4c18-aa79-ba33e53d6392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:01.119956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc809e-a178-4c18-aa79-ba33e53d6392 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:01.120428 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc809e-a178-4c18-aa79-ba33e53d6392 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:01.120717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc809e-a178-4c18-aa79-ba33e53d6392 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:01.132480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc809e-a178-4c18-aa79-ba33e53d6392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:01.134115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc809e-a178-4c18-aa79-ba33e53d6392 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:01.154554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-c2cc809e-a178-4c18-aa79-ba33e53d6392 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:01:01.154554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:01.154554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:01.154554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:01.154554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:01.154554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:01.154554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:01.154554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:01.154554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:01.154554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:01.154554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:01.154554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:01.154554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:01:01.154554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:01:01.154554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:01.155691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:01.156783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:01:01.156783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:01:01.156783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:01:01.156783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:01:01.158339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 537/1074] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:01:01.159750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a332ae-7399-4f72-9144-6fe4ce0776dd None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:01.164726 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc809e-a178-4c18-aa79-ba33e53d6392 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:01.177223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:01.201220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf816e0-38c0-4d2c-9db4-419e69376ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:01.201892 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf816e0-38c0-4d2c-9db4-419e69376ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:01.202179 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf816e0-38c0-4d2c-9db4-419e69376ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:01.202461 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf816e0-38c0-4d2c-9db4-419e69376ed8 None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:01.203289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf816e0-38c0-4d2c-9db4-419e69376ed8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:01.203613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf816e0-38c0-4d2c-9db4-419e69376ed8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:01.205046 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf816e0-38c0-4d2c-9db4-419e69376ed8 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:01.217535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 538/1075] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:01:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:01.223438 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf816e0-38c0-4d2c-9db4-419e69376ed8 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:01.273740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc809e-a178-4c18-aa79-ba33e53d6392 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:01.313817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143bb91-9446-47fc-ae67-16fa22baebcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:01.314499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143bb91-9446-47fc-ae67-16fa22baebcf None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:01.314821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143bb91-9446-47fc-ae67-16fa22baebcf None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:01.315103 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143bb91-9446-47fc-ae67-16fa22baebcf None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:01.315826 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143bb91-9446-47fc-ae67-16fa22baebcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:01.316137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143bb91-9446-47fc-ae67-16fa22baebcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:01.317684 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143bb91-9446-47fc-ae67-16fa22baebcf None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:01.330734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 538/1076] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:01:01.334348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf816e0-38c0-4d2c-9db4-419e69376ed8 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:01.337949 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:01.375536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937176a-9862-4301-8728-4b5ea28c641c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:01.376449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937176a-9862-4301-8728-4b5ea28c641c None admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:01.376736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937176a-9862-4301-8728-4b5ea28c641c None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:01.377018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937176a-9862-4301-8728-4b5ea28c641c None admin] PATH_INFO: `/v3/projects/a498284274454d25b1b118a88996e522/users/be08c43f0d0c4a8194596e54500b44ed/roles/a2636fed60634c3997e3c9213a818395` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:01.379420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937176a-9862-4301-8728-4b5ea28c641c None admin] RBAC: Authorizing `identity:create_grant(project_id=a498284274454d25b1b118a88996e522, user_id=be08c43f0d0c4a8194596e54500b44ed, role_id=a2636fed60634c3997e3c9213a818395)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:01.380904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937176a-9862-4301-8728-4b5ea28c641c None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:01.406712 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 539/1077] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 09:01:01 2022] PUT /identity/v3/projects/a498284274454d25b1b118a88996e522/users/be08c43f0d0c4a8194596e54500b44ed/roles/a2636fed60634c3997e3c9213a818395 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:01:01.720816 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788c8ac-0634-4189-9807-1131b26e9aa4 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:01.721116 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788c8ac-0634-4189-9807-1131b26e9aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:01.721398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788c8ac-0634-4189-9807-1131b26e9aa4 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:01.722811 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 539/1078] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:01.727921 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea43fc-ada7-41d4-8a06-df89252f7b18 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:01.728221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea43fc-ada7-41d4-8a06-df89252f7b18 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:01.728510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea43fc-ada7-41d4-8a06-df89252f7b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:01.745140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-baea43fc-ada7-41d4-8a06-df89252f7b18 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:01.746898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-baea43fc-ada7-41d4-8a06-df89252f7b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:01.812139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 540/1079] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:01.834465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788c8ac-0634-4189-9807-1131b26e9aa4 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:01.902092 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:01.937439 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788c8ac-0634-4189-9807-1131b26e9aa4 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:01.970322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff190e-9163-44ba-8a48-78cc71dea36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:01.971138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff190e-9163-44ba-8a48-78cc71dea36c None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:01.971426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff190e-9163-44ba-8a48-78cc71dea36c None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:01.971705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff190e-9163-44ba-8a48-78cc71dea36c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:01.982284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff190e-9163-44ba-8a48-78cc71dea36c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:01:01.989879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-8bff190e-9163-44ba-8a48-78cc71dea36c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:01:01.989879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:01.989879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:01.989879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:01.989879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:01.989879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:01.989879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:01.989879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:01.989879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:01.989879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:01.989879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:01.989879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:01.989879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:01:01.989879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:01:01.989879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:01:01.989879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:01:01.990838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:01:01.990838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:01:01.990838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:01:01.990838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:01:01.990838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:01.990838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:01.990838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:01.990838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:01.990838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:01.990838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:01.990838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:01.990838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:01.990838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:01.990838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:01.990838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:01.990838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:01.991693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:01.991693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:01.991693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:01.991693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:01.991693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:01:01.991693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:01:01.991693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:01:01.991693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:01:01.991693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:01:01.991693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:01:01.991693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:01:01.991693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:01:01.991693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 540/1080] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:01:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:01:01.997976 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea43fc-ada7-41d4-8a06-df89252f7b18 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:02.088770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:02.096476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea43fc-ada7-41d4-8a06-df89252f7b18 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:02.130871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae375ca-173a-43a0-9c56-6d148342ddb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:02.131533 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae375ca-173a-43a0-9c56-6d148342ddb1 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:02.131816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae375ca-173a-43a0-9c56-6d148342ddb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:02.132091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae375ca-173a-43a0-9c56-6d148342ddb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:02.132839 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae375ca-173a-43a0-9c56-6d148342ddb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:02.133143 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae375ca-173a-43a0-9c56-6d148342ddb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:02.134493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae375ca-173a-43a0-9c56-6d148342ddb1 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:02.142811 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 541/1081] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:01:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:02.148427 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff190e-9163-44ba-8a48-78cc71dea36c None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:02.245992 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff190e-9163-44ba-8a48-78cc71dea36c None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:02.271017 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:02.279503 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ac1f7-3b98-4e69-bdb4-eb73cd58d4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:02.280357 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ac1f7-3b98-4e69-bdb4-eb73cd58d4ef None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:02.280637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ac1f7-3b98-4e69-bdb4-eb73cd58d4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:02.280911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ac1f7-3b98-4e69-bdb4-eb73cd58d4ef None admin] PATH_INFO: `/v3/users/a405801a2c6140949d5725ebb8623700` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:02.282067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ac1f7-3b98-4e69-bdb4-eb73cd58d4ef None admin] RBAC: Authorizing `identity:get_user(user_id=a405801a2c6140949d5725ebb8623700)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:02.283534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ac1f7-3b98-4e69-bdb4-eb73cd58d4ef None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:02.285131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 541/1082] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 09:01:02 2022] GET /identity/v3/users/a405801a2c6140949d5725ebb8623700 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:02.290880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae375ca-173a-43a0-9c56-6d148342ddb1 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:02.390490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae375ca-173a-43a0-9c56-6d148342ddb1 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:02.423242 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca030f4-92c2-4f07-bedd-620594e5793d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:02.424105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca030f4-92c2-4f07-bedd-620594e5793d None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:02.424391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca030f4-92c2-4f07-bedd-620594e5793d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:02.424675 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca030f4-92c2-4f07-bedd-620594e5793d None admin] PATH_INFO: `/v3/projects/e112dfa169e04071b6f410eca228826e` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:02.425843 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca030f4-92c2-4f07-bedd-620594e5793d None admin] RBAC: Authorizing `identity:get_project(project_id=e112dfa169e04071b6f410eca228826e)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:02.427320 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca030f4-92c2-4f07-bedd-620594e5793d None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:02.429017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 542/1083] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 09:01:02 2022] GET /identity/v3/projects/e112dfa169e04071b6f410eca228826e => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:02.435605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ac1f7-3b98-4e69-bdb4-eb73cd58d4ef None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:02.537795 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ac1f7-3b98-4e69-bdb4-eb73cd58d4ef None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:02.575386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616a345-8ba5-4984-833d-aede87a84820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:02.576276 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616a345-8ba5-4984-833d-aede87a84820 None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:02.576562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616a345-8ba5-4984-833d-aede87a84820 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:02.576843 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616a345-8ba5-4984-833d-aede87a84820 None admin] PATH_INFO: `/v3/projects/e112dfa169e04071b6f410eca228826e/users/a405801a2c6140949d5725ebb8623700/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:02.578830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616a345-8ba5-4984-833d-aede87a84820 None admin] RBAC: Authorizing `identity:create_grant(project_id=e112dfa169e04071b6f410eca228826e, user_id=a405801a2c6140949d5725ebb8623700, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:02.580263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616a345-8ba5-4984-833d-aede87a84820 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:02.593674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 542/1084] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 09:01:02 2022] PUT /identity/v3/projects/e112dfa169e04071b6f410eca228826e/users/a405801a2c6140949d5725ebb8623700/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:01:03.450412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9630657-b1d1-4d95-ad23-adfdc554ca46 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:03.450997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9630657-b1d1-4d95-ad23-adfdc554ca46 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:03.450997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9630657-b1d1-4d95-ad23-adfdc554ca46 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:03.452378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 543/1085] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:03.457562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1184e-c336-4e72-90f9-baf503c0e74c None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:03.457952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1184e-c336-4e72-90f9-baf503c0e74c None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:03.458258 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1184e-c336-4e72-90f9-baf503c0e74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:03.475350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-4fb1184e-c336-4e72-90f9-baf503c0e74c None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:03.476518 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4fb1184e-c336-4e72-90f9-baf503c0e74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:03.525048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:03.547167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 543/1086] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:03.570345 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9630657-b1d1-4d95-ad23-adfdc554ca46 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:03.597290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:03.672500 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9630657-b1d1-4d95-ad23-adfdc554ca46 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:03.708468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370258b-f790-4346-b433-8a2424f9e3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:03.709192 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370258b-f790-4346-b433-8a2424f9e3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:03.709477 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370258b-f790-4346-b433-8a2424f9e3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:03.709757 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370258b-f790-4346-b433-8a2424f9e3a3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:03.720376 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370258b-f790-4346-b433-8a2424f9e3a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:01:03.728442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-7370258b-f790-4346-b433-8a2424f9e3a3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:01:03.728442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:03.728442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:03.728442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:03.728442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:03.728442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:03.728442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:03.728442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:03.728442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:03.728442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:03.728442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:03.728442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:03.728442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:01:03.728442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:01:03.728442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:01:03.728442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:01:03.729448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:01:03.729448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:01:03.729448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:01:03.729448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:01:03.729448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:03.729448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:03.729448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:03.729448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:03.729448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:03.729448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:03.729448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:03.729448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:03.729448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:03.729448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:03.729448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:03.729448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:03.730385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:03.730385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:03.730385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:03.730385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:03.730385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:01:03.730385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:01:03.730385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:01:03.730385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:01:03.730385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:01:03.730385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:01:03.730385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 09:01:03.730385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:01:03.730385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 544/1087] 10.222.0.44 () {56 vars in 1120 bytes} [Wed Jul 20 09:01:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:01:03.735649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1184e-c336-4e72-90f9-baf503c0e74c None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:03.832327 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1184e-c336-4e72-90f9-baf503c0e74c None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:03.865592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb07310-37eb-4f03-862f-031afbe5500d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:03.866256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb07310-37eb-4f03-862f-031afbe5500d None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:03.866540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb07310-37eb-4f03-862f-031afbe5500d None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:03.866852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb07310-37eb-4f03-862f-031afbe5500d None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:03.867603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb07310-37eb-4f03-862f-031afbe5500d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:03.867912 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb07310-37eb-4f03-862f-031afbe5500d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:03.869268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb07310-37eb-4f03-862f-031afbe5500d None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:03.877564 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 544/1088] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Jul 20 09:01:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:03.883243 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370258b-f790-4346-b433-8a2424f9e3a3 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:03.964618 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:03.979970 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370258b-f790-4346-b433-8a2424f9e3a3 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:04.013339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449724d2-a3fb-4bba-800e-5d8f7aa35d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:04.014109 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449724d2-a3fb-4bba-800e-5d8f7aa35d08 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:04.014389 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449724d2-a3fb-4bba-800e-5d8f7aa35d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:04.014661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449724d2-a3fb-4bba-800e-5d8f7aa35d08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:04.024556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449724d2-a3fb-4bba-800e-5d8f7aa35d08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:04.025999 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449724d2-a3fb-4bba-800e-5d8f7aa35d08 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:04.044468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-449724d2-a3fb-4bba-800e-5d8f7aa35d08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:01:04.044468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:04.044468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:04.044468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:04.044468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:04.044468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:04.044468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:04.044468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:04.044468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:04.044468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:04.044468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:04.044468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:04.044468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:01:04.044468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:01:04.044468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:04.045378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:04.046290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:01:04.046290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:01:04.046290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:01:04.046290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:01:04.046290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 545/1089] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:01:04.051400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb07310-37eb-4f03-862f-031afbe5500d None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:04.100801 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:04.151494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb07310-37eb-4f03-862f-031afbe5500d None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:04.185912 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10293b19-1765-49b7-bdac-092737e1f831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:04.186574 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10293b19-1765-49b7-bdac-092737e1f831 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:04.186890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10293b19-1765-49b7-bdac-092737e1f831 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:04.187172 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10293b19-1765-49b7-bdac-092737e1f831 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:04.187891 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10293b19-1765-49b7-bdac-092737e1f831 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:04.188202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10293b19-1765-49b7-bdac-092737e1f831 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:04.189569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10293b19-1765-49b7-bdac-092737e1f831 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:04.202001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 545/1090] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:04.207544 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449724d2-a3fb-4bba-800e-5d8f7aa35d08 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:04.309321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449724d2-a3fb-4bba-800e-5d8f7aa35d08 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:04.344418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40a615-303e-4789-ba4b-dbc47e4f3cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:04.345215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40a615-303e-4789-ba4b-dbc47e4f3cda None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:04.345502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40a615-303e-4789-ba4b-dbc47e4f3cda None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:04.345780 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40a615-303e-4789-ba4b-dbc47e4f3cda None admin] PATH_INFO: `/v3/users/nova` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:04.348996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:04.354656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40a615-303e-4789-ba4b-dbc47e4f3cda None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:04.356334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40a615-303e-4789-ba4b-dbc47e4f3cda None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:04.370390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-4c40a615-303e-4789-ba4b-dbc47e4f3cda None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:01:04.370390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:04.370390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:04.370390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:04.370390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:04.370390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:04.370390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:04.370390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:04.370390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:04.370390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:04.370390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:04.370390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:04.370390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:01:04.370390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:01:04.370390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:01:04.370390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:04.371568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:04.372566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:04.372566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:04.372566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:01:04.372566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:01:04.372566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:01:04.372566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:01:04.372566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:01:04.372566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:01:04.372566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 09:01:04.372566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:01:04.372566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 546/1091] 10.222.0.44 () {56 vars in 1093 bytes} [Wed Jul 20 09:01:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:01:04.377641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10293b19-1765-49b7-bdac-092737e1f831 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:04.478506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10293b19-1765-49b7-bdac-092737e1f831 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:04.487648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:04.513067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35770e1-5f17-4938-9a71-62ca067947d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:04.513725 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35770e1-5f17-4938-9a71-62ca067947d6 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:04.514006 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35770e1-5f17-4938-9a71-62ca067947d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:04.514280 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35770e1-5f17-4938-9a71-62ca067947d6 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:04.515135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35770e1-5f17-4938-9a71-62ca067947d6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:04.515445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35770e1-5f17-4938-9a71-62ca067947d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:04.516864 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35770e1-5f17-4938-9a71-62ca067947d6 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:04.533551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 546/1092] 10.222.0.44 () {56 vars in 1133 bytes} [Wed Jul 20 09:01:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:04.539506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40a615-303e-4789-ba4b-dbc47e4f3cda None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:04.607346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6cc07-8e7b-4c03-aa92-95d3ce6bd0b3 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:04.607639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6cc07-8e7b-4c03-aa92-95d3ce6bd0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:04.607920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6cc07-8e7b-4c03-aa92-95d3ce6bd0b3 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:04.609388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 547/1093] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:01:04.614266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa854f1-87be-4dcb-8f6c-12d1c6c1556d None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:04.614553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa854f1-87be-4dcb-8f6c-12d1c6c1556d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:04.614875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa854f1-87be-4dcb-8f6c-12d1c6c1556d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:04.631446 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-8aa854f1-87be-4dcb-8f6c-12d1c6c1556d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:04.632626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8aa854f1-87be-4dcb-8f6c-12d1c6c1556d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:04.641933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40a615-303e-4789-ba4b-dbc47e4f3cda None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:04.674732 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88692460-fd18-4004-9e7c-3b3878fb9674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:04.675383 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88692460-fd18-4004-9e7c-3b3878fb9674 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:04.675917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88692460-fd18-4004-9e7c-3b3878fb9674 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:04.675917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88692460-fd18-4004-9e7c-3b3878fb9674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:04.700538 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 548/1094] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:01:04.724201 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa854f1-87be-4dcb-8f6c-12d1c6c1556d None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:04.736424 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:04.743317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88692460-fd18-4004-9e7c-3b3878fb9674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:04.744781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88692460-fd18-4004-9e7c-3b3878fb9674 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:04.819974 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:04.826024 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa854f1-87be-4dcb-8f6c-12d1c6c1556d None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:04.847279 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 547/1095] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:01:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:01:04.853065 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88692460-fd18-4004-9e7c-3b3878fb9674 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:04.859614 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce69282-1121-4135-866b-2a5a71317be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:04.860345 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce69282-1121-4135-866b-2a5a71317be2 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:04.860627 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce69282-1121-4135-866b-2a5a71317be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:04.861233 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce69282-1121-4135-866b-2a5a71317be2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:04.871677 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce69282-1121-4135-866b-2a5a71317be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:01:04.879279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-4ce69282-1121-4135-866b-2a5a71317be2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:01:04.879279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:04.879279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:04.879279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:04.879279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:04.879279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:04.879279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:04.879279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:04.879279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:04.879279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:04.879279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:04.879279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:04.879279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:01:04.879279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:01:04.879279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:01:04.879279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:01:04.880305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:01:04.880305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:01:04.880305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:01:04.880305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:01:04.880305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:04.880305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:04.880305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:04.880305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:04.880305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:04.880305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:04.880305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:04.880305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:04.880305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:04.880305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:04.880305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:04.880305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:04.881152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:04.881152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:04.881152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:04.881152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:04.881152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:01:04.881152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:01:04.881152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:01:04.881152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:01:04.881152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:01:04.881152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:01:04.881152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:01:04.881152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:01:04.881152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 549/1096] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:01:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:01:04.886351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce69282-1121-4135-866b-2a5a71317be2 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:04.947943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:04.954203 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88692460-fd18-4004-9e7c-3b3878fb9674 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:04.986152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce69282-1121-4135-866b-2a5a71317be2 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:04.989063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639b23d-f144-4b8e-ada7-6e54eeadfef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:04.989841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639b23d-f144-4b8e-ada7-6e54eeadfef7 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:04.990122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639b23d-f144-4b8e-ada7-6e54eeadfef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:04.990390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639b23d-f144-4b8e-ada7-6e54eeadfef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:05.000858 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639b23d-f144-4b8e-ada7-6e54eeadfef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:05.002297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639b23d-f144-4b8e-ada7-6e54eeadfef7 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:05.019670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-c639b23d-f144-4b8e-ada7-6e54eeadfef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:01:05.019670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:05.019670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:05.019670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:05.019670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:05.019670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:05.019670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:05.019670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:05.019670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:05.019670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:05.019670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:05.019670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:05.019670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:01:05.019670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:01:05.019670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:05.020745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:05.021850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:01:05.021850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:01:05.021850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:01:05.021850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:01:05.022123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf1db2-28eb-4678-b6ae-22ef2cf64e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:05.022123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf1db2-28eb-4678-b6ae-22ef2cf64e45 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:05.022123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf1db2-28eb-4678-b6ae-22ef2cf64e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:05.022123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf1db2-28eb-4678-b6ae-22ef2cf64e45 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:05.022449 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf1db2-28eb-4678-b6ae-22ef2cf64e45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:05.022449 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf1db2-28eb-4678-b6ae-22ef2cf64e45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:05.023045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 548/1097] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:01:05.023386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf1db2-28eb-4678-b6ae-22ef2cf64e45 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:05.026558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639b23d-f144-4b8e-ada7-6e54eeadfef7 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:05.036499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 550/1098] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:01:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:01:05.037509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf1db2-28eb-4678-b6ae-22ef2cf64e45 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:05.056740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:05.128794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639b23d-f144-4b8e-ada7-6e54eeadfef7 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:05.139892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf1db2-28eb-4678-b6ae-22ef2cf64e45 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:05.161907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afe3cf-c79a-400c-bd91-584a39fdd316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:05.162555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afe3cf-c79a-400c-bd91-584a39fdd316 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:05.162866 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afe3cf-c79a-400c-bd91-584a39fdd316 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:05.163141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afe3cf-c79a-400c-bd91-584a39fdd316 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:05.163880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afe3cf-c79a-400c-bd91-584a39fdd316 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:05.164182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afe3cf-c79a-400c-bd91-584a39fdd316 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:05.165533 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afe3cf-c79a-400c-bd91-584a39fdd316 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:05.173829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b2c82-8a16-40f6-a4b7-c76aa52ef109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:05.174699 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b2c82-8a16-40f6-a4b7-c76aa52ef109 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:05.174999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b2c82-8a16-40f6-a4b7-c76aa52ef109 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:05.175265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b2c82-8a16-40f6-a4b7-c76aa52ef109 None admin] PATH_INFO: `/v3/users/a405801a2c6140949d5725ebb8623700` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:05.176436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b2c82-8a16-40f6-a4b7-c76aa52ef109 None admin] RBAC: Authorizing `identity:get_user(user_id=a405801a2c6140949d5725ebb8623700)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:05.176870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 549/1099] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:01:05.178079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b2c82-8a16-40f6-a4b7-c76aa52ef109 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:05.179601 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 551/1100] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 09:01:05 2022] GET /identity/v3/users/a405801a2c6140949d5725ebb8623700 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:01:05.182580 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afe3cf-c79a-400c-bd91-584a39fdd316 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:05.185369 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b2c82-8a16-40f6-a4b7-c76aa52ef109 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:05.194335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:05.268452 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:05.282547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afe3cf-c79a-400c-bd91-584a39fdd316 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:05.283193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b2c82-8a16-40f6-a4b7-c76aa52ef109 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:05.316335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5534386-00ba-4e86-b23b-ac29880459d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:05.316517 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df3b60-4704-40cc-86a4-368c5e819c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:05.316978 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5534386-00ba-4e86-b23b-ac29880459d7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:05.317146 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df3b60-4704-40cc-86a4-368c5e819c1b None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:05.317261 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5534386-00ba-4e86-b23b-ac29880459d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:05.317431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df3b60-4704-40cc-86a4-368c5e819c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:05.317522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5534386-00ba-4e86-b23b-ac29880459d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:05.317708 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df3b60-4704-40cc-86a4-368c5e819c1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:05.324831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df3b60-4704-40cc-86a4-368c5e819c1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:05.326264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df3b60-4704-40cc-86a4-368c5e819c1b None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:05.341583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-21df3b60-4704-40cc-86a4-368c5e819c1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:01:05.341583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:05.341583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:05.341583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:05.341583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:05.341583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:05.341583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:05.341583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:05.341583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:05.341583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:05.341583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:05.341583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:05.341583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:01:05.341583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:01:05.341583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:05.342484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:05.343638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:01:05.343638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:01:05.343638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:01:05.343638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:01:05.343638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:01:05.343638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:01:05.343638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:01:05.343638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:01:05.343638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 550/1101] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:01:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:01:05.348523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df3b60-4704-40cc-86a4-368c5e819c1b None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:05.385905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5534386-00ba-4e86-b23b-ac29880459d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:05.387323 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5534386-00ba-4e86-b23b-ac29880459d7 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:05.415762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:05.450254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df3b60-4704-40cc-86a4-368c5e819c1b None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:05.471904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:05.485554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a66b1f-7583-4ef3-a656-4c9ef6f5e8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:05.486211 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a66b1f-7583-4ef3-a656-4c9ef6f5e8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:05.486497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a66b1f-7583-4ef3-a656-4c9ef6f5e8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:05.486806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a66b1f-7583-4ef3-a656-4c9ef6f5e8a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:05.487598 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a66b1f-7583-4ef3-a656-4c9ef6f5e8a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:05.487706 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 552/1102] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:01:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 09:01:05.488118 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a66b1f-7583-4ef3-a656-4c9ef6f5e8a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:05.489326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a66b1f-7583-4ef3-a656-4c9ef6f5e8a6 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:05.493435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5534386-00ba-4e86-b23b-ac29880459d7 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:05.501793 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 551/1103] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:01:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:01:05.507897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a66b1f-7583-4ef3-a656-4c9ef6f5e8a6 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:05.597549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5534386-00ba-4e86-b23b-ac29880459d7 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:05.617131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a66b1f-7583-4ef3-a656-4c9ef6f5e8a6 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:05.633119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb4996-0fc9-49ee-8f8d-f4d497e41a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:05.634005 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb4996-0fc9-49ee-8f8d-f4d497e41a00 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:05.634286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb4996-0fc9-49ee-8f8d-f4d497e41a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:05.634700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb4996-0fc9-49ee-8f8d-f4d497e41a00 None admin] PATH_INFO: `/v3/projects/e112dfa169e04071b6f410eca228826e` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:05.635875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb4996-0fc9-49ee-8f8d-f4d497e41a00 None admin] RBAC: Authorizing `identity:get_project(project_id=e112dfa169e04071b6f410eca228826e)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:05.637544 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb4996-0fc9-49ee-8f8d-f4d497e41a00 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:05.639216 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 553/1104] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 09:01:05 2022] GET /identity/v3/projects/e112dfa169e04071b6f410eca228826e => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:01:05.643192 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:05.645893 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb4996-0fc9-49ee-8f8d-f4d497e41a00 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:05.651280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd85e4c-ffd4-40dd-9152-7dcff623dcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:05.651929 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd85e4c-ffd4-40dd-9152-7dcff623dcff None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:05.652251 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd85e4c-ffd4-40dd-9152-7dcff623dcff None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:05.652619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd85e4c-ffd4-40dd-9152-7dcff623dcff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:05.653484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd85e4c-ffd4-40dd-9152-7dcff623dcff None admin] RBAC: Adding query filter params (role.id=a2636fed60634c3997e3c9213a818395, scope.project.id=a498284274454d25b1b118a88996e522, user.id=be08c43f0d0c4a8194596e54500b44ed) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:05.653792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd85e4c-ffd4-40dd-9152-7dcff623dcff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:05.655268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd85e4c-ffd4-40dd-9152-7dcff623dcff None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:05.661271 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 552/1105] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:01:05 2022] GET /identity/v3/role_assignments?user.id=be08c43f0d0c4a8194596e54500b44ed&scope.project.id=a498284274454d25b1b118a88996e522&role.id=a2636fed60634c3997e3c9213a818395 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:01:05.703786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:05.750617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb4996-0fc9-49ee-8f8d-f4d497e41a00 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:05.786922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a8093-baed-419b-b8b1-8505b5cbc1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:05.787584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a8093-baed-419b-b8b1-8505b5cbc1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:05.787866 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a8093-baed-419b-b8b1-8505b5cbc1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:05.788140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a8093-baed-419b-b8b1-8505b5cbc1c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:05.788969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a8093-baed-419b-b8b1-8505b5cbc1c1 None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=e112dfa169e04071b6f410eca228826e, user.id=a405801a2c6140949d5725ebb8623700) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:05.789279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a8093-baed-419b-b8b1-8505b5cbc1c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:05.790741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a8093-baed-419b-b8b1-8505b5cbc1c1 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:05.797273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 554/1106] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:01:05 2022] GET /identity/v3/role_assignments?user.id=a405801a2c6140949d5725ebb8623700&scope.project.id=e112dfa169e04071b6f410eca228826e&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:01:07.848322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e80fa-86e2-4c39-b884-e771769c24e0 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:07.848847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e80fa-86e2-4c39-b884-e771769c24e0 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:07.848949 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e80fa-86e2-4c39-b884-e771769c24e0 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:07.850238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 553/1107] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:07.855325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08c822-323e-40a5-bf65-e11dcf759241 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:07.855828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08c822-323e-40a5-bf65-e11dcf759241 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:07.855940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08c822-323e-40a5-bf65-e11dcf759241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:07.871978 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-db08c822-323e-40a5-bf65-e11dcf759241 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:07.873140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-db08c822-323e-40a5-bf65-e11dcf759241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:07.937428 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 555/1108] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:07.963294 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e80fa-86e2-4c39-b884-e771769c24e0 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:08.061841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e80fa-86e2-4c39-b884-e771769c24e0 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:08.094998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8372016-dfb8-4383-98a8-5d8101e7c906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:08.095785 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8372016-dfb8-4383-98a8-5d8101e7c906 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:08.096074 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8372016-dfb8-4383-98a8-5d8101e7c906 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:08.096354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8372016-dfb8-4383-98a8-5d8101e7c906 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:08.097519 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8372016-dfb8-4383-98a8-5d8101e7c906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:08.099023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8372016-dfb8-4383-98a8-5d8101e7c906 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:08.100580 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 554/1109] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:01:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:08.106622 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08c822-323e-40a5-bf65-e11dcf759241 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:08.155354 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:08.206615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08c822-323e-40a5-bf65-e11dcf759241 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:08.239760 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3b34e-3957-44ea-b707-d43a0d6b1626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:08.240417 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3b34e-3957-44ea-b707-d43a0d6b1626 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:08.240701 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3b34e-3957-44ea-b707-d43a0d6b1626 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:08.240977 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3b34e-3957-44ea-b707-d43a0d6b1626 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:08.241674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3b34e-3957-44ea-b707-d43a0d6b1626 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:08.243122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3b34e-3957-44ea-b707-d43a0d6b1626 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:08.264418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 556/1110] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 09:01:08 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:01:10.300533 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ba7f7-74a6-4a90-8aad-4c23d44b1efc None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:10.301184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ba7f7-74a6-4a90-8aad-4c23d44b1efc None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:10.301184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ba7f7-74a6-4a90-8aad-4c23d44b1efc None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:10.302481 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 555/1111] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:10.307525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e398e15-2453-4170-936a-afbd0041789a None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:10.307927 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e398e15-2453-4170-936a-afbd0041789a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:10.308084 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e398e15-2453-4170-936a-afbd0041789a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:10.325250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-6e398e15-2453-4170-936a-afbd0041789a None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:10.326408 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6e398e15-2453-4170-936a-afbd0041789a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:10.390072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 557/1112] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:10.414122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ba7f7-74a6-4a90-8aad-4c23d44b1efc None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:10.473831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:10.515522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ba7f7-74a6-4a90-8aad-4c23d44b1efc None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:10.547135 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f5e2b-c844-4cdd-b73d-e9d67c0264c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:10.547999 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f5e2b-c844-4cdd-b73d-e9d67c0264c8 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:10.548279 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f5e2b-c844-4cdd-b73d-e9d67c0264c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:10.548552 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f5e2b-c844-4cdd-b73d-e9d67c0264c8 None admin] PATH_INFO: `/v3/roles/05ca4843c34e463c94de09f756d95318` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:10.550035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f5e2b-c844-4cdd-b73d-e9d67c0264c8 None admin] RBAC: Authorizing `identity:get_role(role_id=05ca4843c34e463c94de09f756d95318)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:10.551457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f5e2b-c844-4cdd-b73d-e9d67c0264c8 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:10.552784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 556/1113] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 09:01:10 2022] GET /identity/v3/roles/05ca4843c34e463c94de09f756d95318 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:10.559895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e398e15-2453-4170-936a-afbd0041789a None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:10.593806 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:10.660649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e398e15-2453-4170-936a-afbd0041789a None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:10.693892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dedb2e-6e3a-4ffc-a406-e323ad592e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:10.695032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dedb2e-6e3a-4ffc-a406-e323ad592e4f None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:10.695324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dedb2e-6e3a-4ffc-a406-e323ad592e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:10.695605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dedb2e-6e3a-4ffc-a406-e323ad592e4f None admin] PATH_INFO: `/v3/users/d57a752a12d0472d9286dbc976560822` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:10.709457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dedb2e-6e3a-4ffc-a406-e323ad592e4f None admin] RBAC: Authorizing `identity:get_user(user_id=d57a752a12d0472d9286dbc976560822)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:10.710923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dedb2e-6e3a-4ffc-a406-e323ad592e4f None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:10.712543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 558/1114] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 09:01:10 2022] GET /identity/v3/users/d57a752a12d0472d9286dbc976560822 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:10.718610 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f5e2b-c844-4cdd-b73d-e9d67c0264c8 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:10.815918 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f5e2b-c844-4cdd-b73d-e9d67c0264c8 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:10.840766 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:10.848704 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab250ec-844f-4c60-b6e1-60ddb7793563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:10.849341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab250ec-844f-4c60-b6e1-60ddb7793563 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:10.849623 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab250ec-844f-4c60-b6e1-60ddb7793563 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:10.849899 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab250ec-844f-4c60-b6e1-60ddb7793563 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:10.914671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab250ec-844f-4c60-b6e1-60ddb7793563 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:10.916161 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab250ec-844f-4c60-b6e1-60ddb7793563 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:11.012770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 557/1115] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:01:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 09:01:11.018758 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dedb2e-6e3a-4ffc-a406-e323ad592e4f None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:11.108429 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:11.117097 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dedb2e-6e3a-4ffc-a406-e323ad592e4f None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:11.150176 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705d743-2bd0-4d85-8b15-444964407361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:11.151054 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705d743-2bd0-4d85-8b15-444964407361 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:11.151344 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705d743-2bd0-4d85-8b15-444964407361 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:11.151626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705d743-2bd0-4d85-8b15-444964407361 None admin] PATH_INFO: `/v3/projects/e112dfa169e04071b6f410eca228826e` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:11.153347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705d743-2bd0-4d85-8b15-444964407361 None admin] RBAC: Authorizing `identity:get_project(project_id=e112dfa169e04071b6f410eca228826e)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:11.154791 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705d743-2bd0-4d85-8b15-444964407361 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:11.156463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 559/1116] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 09:01:11 2022] GET /identity/v3/projects/e112dfa169e04071b6f410eca228826e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:11.162650 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab250ec-844f-4c60-b6e1-60ddb7793563 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:11.189963 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:11.259895 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab250ec-844f-4c60-b6e1-60ddb7793563 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:11.294095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa25c7-9d29-404c-a74a-55c35d307bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:11.294765 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa25c7-9d29-404c-a74a-55c35d307bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:11.295051 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa25c7-9d29-404c-a74a-55c35d307bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:11.295329 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa25c7-9d29-404c-a74a-55c35d307bd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:11.296152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa25c7-9d29-404c-a74a-55c35d307bd8 None admin] RBAC: Adding query filter params (role.id=05ca4843c34e463c94de09f756d95318, scope.project.id=e112dfa169e04071b6f410eca228826e, user.id=d57a752a12d0472d9286dbc976560822) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:11.296462 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa25c7-9d29-404c-a74a-55c35d307bd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:11.297880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa25c7-9d29-404c-a74a-55c35d307bd8 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:11.303277 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 558/1117] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:01:11 2022] GET /identity/v3/role_assignments?user.id=d57a752a12d0472d9286dbc976560822&scope.project.id=e112dfa169e04071b6f410eca228826e&role.id=05ca4843c34e463c94de09f756d95318 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:13.416545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc838a-7e6d-4b46-9b99-c0afc7327a97 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:13.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc838a-7e6d-4b46-9b99-c0afc7327a97 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:13.418467 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc838a-7e6d-4b46-9b99-c0afc7327a97 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:13.418655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 560/1118] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:13.423837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610e218-05a5-4407-b425-09a300c3247e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:13.424291 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610e218-05a5-4407-b425-09a300c3247e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:13.424391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610e218-05a5-4407-b425-09a300c3247e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:13.441193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-7610e218-05a5-4407-b425-09a300c3247e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:13.442366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7610e218-05a5-4407-b425-09a300c3247e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:13.505362 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 559/1119] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:13.528299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc838a-7e6d-4b46-9b99-c0afc7327a97 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:13.628078 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc838a-7e6d-4b46-9b99-c0afc7327a97 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:13.660271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e492e1-16ac-4a8a-9dd8-680bff01abf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:13.661103 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e492e1-16ac-4a8a-9dd8-680bff01abf1 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:13.661384 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e492e1-16ac-4a8a-9dd8-680bff01abf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:13.661661 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e492e1-16ac-4a8a-9dd8-680bff01abf1 None admin] PATH_INFO: `/v3/roles/05ca4843c34e463c94de09f756d95318` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:13.663121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e492e1-16ac-4a8a-9dd8-680bff01abf1 None admin] RBAC: Authorizing `identity:get_role(role_id=05ca4843c34e463c94de09f756d95318)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:13.664475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e492e1-16ac-4a8a-9dd8-680bff01abf1 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:13.665822 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 561/1120] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 09:01:13 2022] GET /identity/v3/roles/05ca4843c34e463c94de09f756d95318 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:13.671819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610e218-05a5-4407-b425-09a300c3247e None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:13.695493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:13.768820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610e218-05a5-4407-b425-09a300c3247e None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:13.801616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68b617-0c86-41f7-8087-4edb0d69cb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:13.802481 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68b617-0c86-41f7-8087-4edb0d69cb6a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:13.802797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68b617-0c86-41f7-8087-4edb0d69cb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:13.803080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68b617-0c86-41f7-8087-4edb0d69cb6a None admin] PATH_INFO: `/v3/users/d57a752a12d0472d9286dbc976560822` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:13.804260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68b617-0c86-41f7-8087-4edb0d69cb6a None admin] RBAC: Authorizing `identity:get_user(user_id=d57a752a12d0472d9286dbc976560822)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:13.805675 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68b617-0c86-41f7-8087-4edb0d69cb6a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:13.807332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 560/1121] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 09:01:13 2022] GET /identity/v3/users/d57a752a12d0472d9286dbc976560822 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:13.813720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e492e1-16ac-4a8a-9dd8-680bff01abf1 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:13.840905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:13.908552 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e492e1-16ac-4a8a-9dd8-680bff01abf1 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:13.941348 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603ff66-5fd7-402d-a660-491f74f2176c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:13.942121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603ff66-5fd7-402d-a660-491f74f2176c None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:13.942401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603ff66-5fd7-402d-a660-491f74f2176c None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:13.942700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603ff66-5fd7-402d-a660-491f74f2176c None admin] PATH_INFO: `/v3/projects/e112dfa169e04071b6f410eca228826e` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:13.943847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603ff66-5fd7-402d-a660-491f74f2176c None admin] RBAC: Authorizing `identity:get_project(project_id=e112dfa169e04071b6f410eca228826e)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:13.945500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603ff66-5fd7-402d-a660-491f74f2176c None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:13.947163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 562/1122] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 09:01:13 2022] GET /identity/v3/projects/e112dfa169e04071b6f410eca228826e => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:13.953463 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68b617-0c86-41f7-8087-4edb0d69cb6a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:14.040867 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:14.051343 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68b617-0c86-41f7-8087-4edb0d69cb6a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:14.084598 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de52a8b-057f-4ee4-a43b-9ea77bf225be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:14.085477 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de52a8b-057f-4ee4-a43b-9ea77bf225be None admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:14.085759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de52a8b-057f-4ee4-a43b-9ea77bf225be None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:14.086036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de52a8b-057f-4ee4-a43b-9ea77bf225be None admin] PATH_INFO: `/v3/projects/e112dfa169e04071b6f410eca228826e/users/d57a752a12d0472d9286dbc976560822/roles/05ca4843c34e463c94de09f756d95318` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:14.088231 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de52a8b-057f-4ee4-a43b-9ea77bf225be None admin] RBAC: Authorizing `identity:create_grant(project_id=e112dfa169e04071b6f410eca228826e, user_id=d57a752a12d0472d9286dbc976560822, role_id=05ca4843c34e463c94de09f756d95318)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:14.089714 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de52a8b-057f-4ee4-a43b-9ea77bf225be None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:14.101635 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 561/1123] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 09:01:13 2022] PUT /identity/v3/projects/e112dfa169e04071b6f410eca228826e/users/d57a752a12d0472d9286dbc976560822/roles/05ca4843c34e463c94de09f756d95318 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:01:16.113079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6e55f-be02-4835-b0ba-2dffcfff6485 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:16.113607 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6e55f-be02-4835-b0ba-2dffcfff6485 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:16.113698 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6e55f-be02-4835-b0ba-2dffcfff6485 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:16.115056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 563/1124] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:16.120138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23eb468-91a6-406f-919d-65a25783d195 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:16.121117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23eb468-91a6-406f-919d-65a25783d195 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:16.121389 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23eb468-91a6-406f-919d-65a25783d195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:16.137676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-d23eb468-91a6-406f-919d-65a25783d195 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:16.138898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d23eb468-91a6-406f-919d-65a25783d195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:16.204208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 562/1125] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:16.227469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6e55f-be02-4835-b0ba-2dffcfff6485 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:16.321152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:16.328669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6e55f-be02-4835-b0ba-2dffcfff6485 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:16.364182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0ab50-d6d9-4629-a93c-3c90acc21308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:16.364962 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0ab50-d6d9-4629-a93c-3c90acc21308 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:16.365241 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0ab50-d6d9-4629-a93c-3c90acc21308 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:16.365517 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0ab50-d6d9-4629-a93c-3c90acc21308 None admin] PATH_INFO: `/v3/roles/05ca4843c34e463c94de09f756d95318` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:16.367097 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0ab50-d6d9-4629-a93c-3c90acc21308 None admin] RBAC: Authorizing `identity:get_role(role_id=05ca4843c34e463c94de09f756d95318)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:16.368636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0ab50-d6d9-4629-a93c-3c90acc21308 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:16.370183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 564/1126] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 09:01:16 2022] GET /identity/v3/roles/05ca4843c34e463c94de09f756d95318 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:16.376619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23eb468-91a6-406f-919d-65a25783d195 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:16.456480 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:16.477285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23eb468-91a6-406f-919d-65a25783d195 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:16.511505 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185b211-6cc1-45cd-b233-820ff3d7cda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:16.512381 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185b211-6cc1-45cd-b233-820ff3d7cda3 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:16.512665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185b211-6cc1-45cd-b233-820ff3d7cda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:16.512942 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185b211-6cc1-45cd-b233-820ff3d7cda3 None admin] PATH_INFO: `/v3/users/d57a752a12d0472d9286dbc976560822` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:16.514142 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185b211-6cc1-45cd-b233-820ff3d7cda3 None admin] RBAC: Authorizing `identity:get_user(user_id=d57a752a12d0472d9286dbc976560822)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:16.515599 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185b211-6cc1-45cd-b233-820ff3d7cda3 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:16.517445 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 563/1127] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 09:01:16 2022] GET /identity/v3/users/d57a752a12d0472d9286dbc976560822 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:16.524165 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0ab50-d6d9-4629-a93c-3c90acc21308 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:16.627879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0ab50-d6d9-4629-a93c-3c90acc21308 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:16.661021 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfedbf7-f956-4186-aa2a-2ff97368122a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:16.661667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfedbf7-f956-4186-aa2a-2ff97368122a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:16.661952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfedbf7-f956-4186-aa2a-2ff97368122a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:16.662235 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfedbf7-f956-4186-aa2a-2ff97368122a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:16.691990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:16.728203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfedbf7-f956-4186-aa2a-2ff97368122a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:16.729648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfedbf7-f956-4186-aa2a-2ff97368122a None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:16.824171 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 565/1128] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:01:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:01:16.830212 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185b211-6cc1-45cd-b233-820ff3d7cda3 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:16.928485 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185b211-6cc1-45cd-b233-820ff3d7cda3 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:16.962518 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455969b-8030-4966-9328-f0fe671923cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:16.963400 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455969b-8030-4966-9328-f0fe671923cc None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:16.963681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455969b-8030-4966-9328-f0fe671923cc None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:16.963955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455969b-8030-4966-9328-f0fe671923cc None admin] PATH_INFO: `/v3/projects/e112dfa169e04071b6f410eca228826e` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:16.965931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455969b-8030-4966-9328-f0fe671923cc None admin] RBAC: Authorizing `identity:get_project(project_id=e112dfa169e04071b6f410eca228826e)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:16.967408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455969b-8030-4966-9328-f0fe671923cc None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:16.969087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 564/1129] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 09:01:16 2022] GET /identity/v3/projects/e112dfa169e04071b6f410eca228826e => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:16.975204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfedbf7-f956-4186-aa2a-2ff97368122a None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:17.035182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:17.071132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfedbf7-f956-4186-aa2a-2ff97368122a None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:17.105037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db47be-bf41-44a9-a5a1-4a73df7187fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:17.105674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db47be-bf41-44a9-a5a1-4a73df7187fc None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:17.105955 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db47be-bf41-44a9-a5a1-4a73df7187fc None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:17.106229 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db47be-bf41-44a9-a5a1-4a73df7187fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:17.107075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db47be-bf41-44a9-a5a1-4a73df7187fc None admin] RBAC: Adding query filter params (role.id=05ca4843c34e463c94de09f756d95318, scope.project.id=e112dfa169e04071b6f410eca228826e, user.id=d57a752a12d0472d9286dbc976560822) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:17.107383 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db47be-bf41-44a9-a5a1-4a73df7187fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:17.108785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db47be-bf41-44a9-a5a1-4a73df7187fc None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:17.114338 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 566/1130] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:01:16 2022] GET /identity/v3/role_assignments?user.id=d57a752a12d0472d9286dbc976560822&scope.project.id=e112dfa169e04071b6f410eca228826e&role.id=05ca4843c34e463c94de09f756d95318 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:19.155709 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430c54d-51f9-4ae0-84fd-8868d52416a5 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:19.156261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430c54d-51f9-4ae0-84fd-8868d52416a5 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:19.156261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430c54d-51f9-4ae0-84fd-8868d52416a5 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:19.157619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 565/1131] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:19.162784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd4f98-8975-4818-9964-622313eb3e4a None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:19.163168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd4f98-8975-4818-9964-622313eb3e4a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:19.163342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd4f98-8975-4818-9964-622313eb3e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:19.180411 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-eecd4f98-8975-4818-9964-622313eb3e4a None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:19.181585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-eecd4f98-8975-4818-9964-622313eb3e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:19.248307 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 567/1132] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:19.271949 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430c54d-51f9-4ae0-84fd-8868d52416a5 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:19.289724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:19.371873 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430c54d-51f9-4ae0-84fd-8868d52416a5 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:19.404564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc641841-959f-4edd-8247-b9faf70751e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:19.405333 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc641841-959f-4edd-8247-b9faf70751e7 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:19.405610 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc641841-959f-4edd-8247-b9faf70751e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:19.405929 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc641841-959f-4edd-8247-b9faf70751e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:19.407052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc641841-959f-4edd-8247-b9faf70751e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:19.408473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc641841-959f-4edd-8247-b9faf70751e7 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:19.410045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 566/1133] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:01:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:19.415938 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd4f98-8975-4818-9964-622313eb3e4a None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:19.473319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:19.514807 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd4f98-8975-4818-9964-622313eb3e4a None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:19.548728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8cb4c-66a3-4068-bb32-2bdc556ebb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:19.549389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8cb4c-66a3-4068-bb32-2bdc556ebb51 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:19.549672 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8cb4c-66a3-4068-bb32-2bdc556ebb51 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:19.549949 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8cb4c-66a3-4068-bb32-2bdc556ebb51 None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:19.550691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8cb4c-66a3-4068-bb32-2bdc556ebb51 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:19.552129 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8cb4c-66a3-4068-bb32-2bdc556ebb51 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:19.578529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 568/1134] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jul 20 09:01:19 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:01:21.617945 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa164ae7-109d-48b9-a9d9-bfbf9ac80a6a None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:21.618487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa164ae7-109d-48b9-a9d9-bfbf9ac80a6a None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:21.618487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa164ae7-109d-48b9-a9d9-bfbf9ac80a6a None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:21.619912 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 567/1135] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:21.625719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f444d-ea11-4bec-8f2e-9321dffc4636 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:21.626149 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f444d-ea11-4bec-8f2e-9321dffc4636 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:21.626287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f444d-ea11-4bec-8f2e-9321dffc4636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:21.643223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-4c8f444d-ea11-4bec-8f2e-9321dffc4636 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:21.644409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4c8f444d-ea11-4bec-8f2e-9321dffc4636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:21.709481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 569/1136] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:21.734318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa164ae7-109d-48b9-a9d9-bfbf9ac80a6a None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:21.816781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:21.833471 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa164ae7-109d-48b9-a9d9-bfbf9ac80a6a None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:21.866319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63724e41-d22f-4ed2-b5ae-49ad0ec0a279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:21.867125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63724e41-d22f-4ed2-b5ae-49ad0ec0a279 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:21.867413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63724e41-d22f-4ed2-b5ae-49ad0ec0a279 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:21.867694 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63724e41-d22f-4ed2-b5ae-49ad0ec0a279 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:21.868854 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63724e41-d22f-4ed2-b5ae-49ad0ec0a279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:21.870282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63724e41-d22f-4ed2-b5ae-49ad0ec0a279 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:21.872434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 568/1137] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:01:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:21.878032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f444d-ea11-4bec-8f2e-9321dffc4636 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:21.917254 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:21.973147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f444d-ea11-4bec-8f2e-9321dffc4636 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:22.005849 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ecdd2-941a-407f-9858-7c5f5e600b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:22.006499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ecdd2-941a-407f-9858-7c5f5e600b04 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:22.006811 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ecdd2-941a-407f-9858-7c5f5e600b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:22.007088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ecdd2-941a-407f-9858-7c5f5e600b04 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:22.007782 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ecdd2-941a-407f-9858-7c5f5e600b04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:22.009193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ecdd2-941a-407f-9858-7c5f5e600b04 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:22.027926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 570/1138] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 09:01:21 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:01:24.069202 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb66add-ea83-4617-a622-ae102c9f8424 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:24.070022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb66add-ea83-4617-a622-ae102c9f8424 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:24.070022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb66add-ea83-4617-a622-ae102c9f8424 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:24.071425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 569/1139] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:24.076796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94df9c-6657-4c77-98fd-90d4397f8c22 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:24.077187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94df9c-6657-4c77-98fd-90d4397f8c22 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:24.077357 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94df9c-6657-4c77-98fd-90d4397f8c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:24.094069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-aa94df9c-6657-4c77-98fd-90d4397f8c22 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:24.095272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-aa94df9c-6657-4c77-98fd-90d4397f8c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:24.159759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 571/1140] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:01:24.183586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb66add-ea83-4617-a622-ae102c9f8424 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:24.282037 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb66add-ea83-4617-a622-ae102c9f8424 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:24.313792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf24dba-4030-4643-afab-7bd5e1649b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:24.314564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf24dba-4030-4643-afab-7bd5e1649b0a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:24.314870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf24dba-4030-4643-afab-7bd5e1649b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:24.315151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf24dba-4030-4643-afab-7bd5e1649b0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:24.325829 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf24dba-4030-4643-afab-7bd5e1649b0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:01:24.333282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-caf24dba-4030-4643-afab-7bd5e1649b0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:01:24.333282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:24.333282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:24.333282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:24.333282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:24.333282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:24.333282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:24.333282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:24.333282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:24.333282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:24.333282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:24.333282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:24.333282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:01:24.333282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:01:24.333282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:01:24.333282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:01:24.334418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:01:24.334418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:01:24.334418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:01:24.334418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:01:24.334418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:24.334418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:24.334418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:24.334418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:24.334418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:24.334418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:24.334418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:24.334418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:24.334418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:24.334418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:24.334418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:24.334418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:24.335396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:24.335396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:24.335396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:24.335396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:24.335396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:01:24.335396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:01:24.335396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:01:24.335396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:01:24.335396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:01:24.335396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:01:24.335396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:01:24.335396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:01:24.335396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 570/1141] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:01:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:01:24.341164 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94df9c-6657-4c77-98fd-90d4397f8c22 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:24.366739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:24.436945 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94df9c-6657-4c77-98fd-90d4397f8c22 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:24.471379 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf0147-75a1-4203-97f9-8cee225e39e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:24.472033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0147-75a1-4203-97f9-8cee225e39e5 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:24.472315 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0147-75a1-4203-97f9-8cee225e39e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:24.472592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf0147-75a1-4203-97f9-8cee225e39e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:24.473336 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf0147-75a1-4203-97f9-8cee225e39e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:24.473649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf0147-75a1-4203-97f9-8cee225e39e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:24.475032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf0147-75a1-4203-97f9-8cee225e39e5 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:24.483125 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 572/1142] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:01:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:24.488800 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf24dba-4030-4643-afab-7bd5e1649b0a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:24.503723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:24.587725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf24dba-4030-4643-afab-7bd5e1649b0a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:24.620702 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0401a-f3db-4458-8230-14d0ffbd25aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:24.622089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0401a-f3db-4458-8230-14d0ffbd25aa None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:24.622363 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0401a-f3db-4458-8230-14d0ffbd25aa None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:24.622636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0401a-f3db-4458-8230-14d0ffbd25aa None admin] PATH_INFO: `/v3/users/0557a34b19c343da85e41dd0d8b4c1ff` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:24.635784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0401a-f3db-4458-8230-14d0ffbd25aa None admin] RBAC: Authorizing `identity:get_user(user_id=0557a34b19c343da85e41dd0d8b4c1ff)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:24.637840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d0401a-f3db-4458-8230-14d0ffbd25aa None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:24.639526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 571/1143] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 09:01:24 2022] GET /identity/v3/users/0557a34b19c343da85e41dd0d8b4c1ff => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:24.645194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf0147-75a1-4203-97f9-8cee225e39e5 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:24.732239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:24.744135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf0147-75a1-4203-97f9-8cee225e39e5 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:24.779216 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8446f55-1ceb-4fda-b013-a72584459473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:24.779852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8446f55-1ceb-4fda-b013-a72584459473 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:24.780137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8446f55-1ceb-4fda-b013-a72584459473 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:24.780413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8446f55-1ceb-4fda-b013-a72584459473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:24.846808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8446f55-1ceb-4fda-b013-a72584459473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:24.848200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8446f55-1ceb-4fda-b013-a72584459473 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:24.932203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:24.945397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 573/1144] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:01:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:01:24.951266 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0401a-f3db-4458-8230-14d0ffbd25aa None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:25.018492 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:25.049094 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0401a-f3db-4458-8230-14d0ffbd25aa None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:25.081933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b92059-6f53-48ab-98a1-d1b17f79b289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:25.082824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b92059-6f53-48ab-98a1-d1b17f79b289 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:25.083107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b92059-6f53-48ab-98a1-d1b17f79b289 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:25.083385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b92059-6f53-48ab-98a1-d1b17f79b289 None admin] PATH_INFO: `/v3/projects/b2b390833758448288f09da36a6822c8` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:25.084557 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b92059-6f53-48ab-98a1-d1b17f79b289 None admin] RBAC: Authorizing `identity:get_project(project_id=b2b390833758448288f09da36a6822c8)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:25.085986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b92059-6f53-48ab-98a1-d1b17f79b289 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:25.087696 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 572/1145] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 09:01:24 2022] GET /identity/v3/projects/b2b390833758448288f09da36a6822c8 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:25.093767 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8446f55-1ceb-4fda-b013-a72584459473 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:25.189171 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8446f55-1ceb-4fda-b013-a72584459473 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:25.221125 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3ade0-bbdf-4752-a35d-135a1d9997e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:25.221768 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3ade0-bbdf-4752-a35d-135a1d9997e7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:25.222047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3ade0-bbdf-4752-a35d-135a1d9997e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:25.222321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3ade0-bbdf-4752-a35d-135a1d9997e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:25.223161 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3ade0-bbdf-4752-a35d-135a1d9997e7 None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=b2b390833758448288f09da36a6822c8, user.id=0557a34b19c343da85e41dd0d8b4c1ff) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:25.223471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3ade0-bbdf-4752-a35d-135a1d9997e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:25.224871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3ade0-bbdf-4752-a35d-135a1d9997e7 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:25.230030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 574/1146] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:01:25 2022] GET /identity/v3/role_assignments?user.id=0557a34b19c343da85e41dd0d8b4c1ff&scope.project.id=b2b390833758448288f09da36a6822c8&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:27.228070 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883f533-650f-4120-90bf-aedf049422f8 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:27.228614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883f533-650f-4120-90bf-aedf049422f8 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:27.228614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883f533-650f-4120-90bf-aedf049422f8 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:27.230003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 573/1147] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:27.235927 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db043fe-db8b-43a1-9cb2-fac14321c63d None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:27.236353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db043fe-db8b-43a1-9cb2-fac14321c63d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:27.236493 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db043fe-db8b-43a1-9cb2-fac14321c63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:27.253394 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-5db043fe-db8b-43a1-9cb2-fac14321c63d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:27.254572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5db043fe-db8b-43a1-9cb2-fac14321c63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:27.300559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:27.321189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 575/1148] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:27.343311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883f533-650f-4120-90bf-aedf049422f8 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:27.442914 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883f533-650f-4120-90bf-aedf049422f8 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:27.475249 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84f8a6-c536-47b2-bbf2-b6b61af13184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:27.476023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84f8a6-c536-47b2-bbf2-b6b61af13184 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:27.476306 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84f8a6-c536-47b2-bbf2-b6b61af13184 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:27.476582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84f8a6-c536-47b2-bbf2-b6b61af13184 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:27.483816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:27.487725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84f8a6-c536-47b2-bbf2-b6b61af13184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:01:27.495456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-5d84f8a6-c536-47b2-bbf2-b6b61af13184 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:01:27.495456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:27.495456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:27.495456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:27.495456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:27.495456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:27.495456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:27.495456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:27.495456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:27.495456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:27.495456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:27.495456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:27.495456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:01:27.495456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:01:27.495456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:01:27.495456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:01:27.497358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:01:27.497358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:01:27.497358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:01:27.497358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:01:27.497358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:27.497358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:27.497358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:27.497358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:27.497358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:27.497358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:27.497358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:27.497358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:27.497358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:27.497358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:27.497358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:27.497358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:27.498401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:27.498401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:27.498401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:27.498401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:27.498401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:01:27.498401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:01:27.498401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:01:27.498401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:01:27.498401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:01:27.498401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:01:27.498401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:01:27.498401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:01:27.498401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 574/1149] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:01:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:01:27.503364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db043fe-db8b-43a1-9cb2-fac14321c63d None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:27.597727 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db043fe-db8b-43a1-9cb2-fac14321c63d None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:27.630670 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ea2e6-c5f0-45ca-a8e2-314dafa3af21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:27.631345 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ea2e6-c5f0-45ca-a8e2-314dafa3af21 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:27.631626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ea2e6-c5f0-45ca-a8e2-314dafa3af21 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:27.631899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ea2e6-c5f0-45ca-a8e2-314dafa3af21 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:27.632636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ea2e6-c5f0-45ca-a8e2-314dafa3af21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:27.632938 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ea2e6-c5f0-45ca-a8e2-314dafa3af21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:27.634286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ea2e6-c5f0-45ca-a8e2-314dafa3af21 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:27.642031 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 576/1150] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:01:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:27.648684 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84f8a6-c536-47b2-bbf2-b6b61af13184 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:27.751390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84f8a6-c536-47b2-bbf2-b6b61af13184 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:27.784112 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e8396-20d2-4b29-b012-f8a1e52cdef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:27.784969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e8396-20d2-4b29-b012-f8a1e52cdef1 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:27.785256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e8396-20d2-4b29-b012-f8a1e52cdef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:27.785531 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e8396-20d2-4b29-b012-f8a1e52cdef1 None admin] PATH_INFO: `/v3/users/0557a34b19c343da85e41dd0d8b4c1ff` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:27.786714 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e8396-20d2-4b29-b012-f8a1e52cdef1 None admin] RBAC: Authorizing `identity:get_user(user_id=0557a34b19c343da85e41dd0d8b4c1ff)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:27.788147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e8396-20d2-4b29-b012-f8a1e52cdef1 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:27.789751 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 575/1151] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 09:01:27 2022] GET /identity/v3/users/0557a34b19c343da85e41dd0d8b4c1ff => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:27.796724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ea2e6-c5f0-45ca-a8e2-314dafa3af21 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:27.841680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:27.893840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ea2e6-c5f0-45ca-a8e2-314dafa3af21 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:27.926069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc25a6-92d2-42cb-8021-7913908dfbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:27.926931 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc25a6-92d2-42cb-8021-7913908dfbd7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:27.927212 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc25a6-92d2-42cb-8021-7913908dfbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:27.927484 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc25a6-92d2-42cb-8021-7913908dfbd7 None admin] PATH_INFO: `/v3/projects/b2b390833758448288f09da36a6822c8` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:27.928637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc25a6-92d2-42cb-8021-7913908dfbd7 None admin] RBAC: Authorizing `identity:get_project(project_id=b2b390833758448288f09da36a6822c8)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:27.930044 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc25a6-92d2-42cb-8021-7913908dfbd7 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:27.931718 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 577/1152] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 09:01:27 2022] GET /identity/v3/projects/b2b390833758448288f09da36a6822c8 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:27.937548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e8396-20d2-4b29-b012-f8a1e52cdef1 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:28.001069 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:28.036052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e8396-20d2-4b29-b012-f8a1e52cdef1 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:28.067542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e849b-6667-4c60-ac27-b1e9808932a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:28.068414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e849b-6667-4c60-ac27-b1e9808932a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:28.068696 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e849b-6667-4c60-ac27-b1e9808932a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:28.068977 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e849b-6667-4c60-ac27-b1e9808932a6 None admin] PATH_INFO: `/v3/projects/b2b390833758448288f09da36a6822c8/users/0557a34b19c343da85e41dd0d8b4c1ff/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:28.071183 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e849b-6667-4c60-ac27-b1e9808932a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2b390833758448288f09da36a6822c8, user_id=0557a34b19c343da85e41dd0d8b4c1ff, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:28.072597 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e849b-6667-4c60-ac27-b1e9808932a6 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:28.085291 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 576/1153] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 09:01:27 2022] PUT /identity/v3/projects/b2b390833758448288f09da36a6822c8/users/0557a34b19c343da85e41dd0d8b4c1ff/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:01:30.068847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69850690-aa2b-497c-9968-004ac87c1ac9 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:30.069378 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69850690-aa2b-497c-9968-004ac87c1ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:30.069520 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69850690-aa2b-497c-9968-004ac87c1ac9 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:30.071051 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 578/1154] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:30.076660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc45da-3289-4633-b77b-d2ccc852cf9c None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:30.077074 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc45da-3289-4633-b77b-d2ccc852cf9c None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:30.077222 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc45da-3289-4633-b77b-d2ccc852cf9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:30.093780 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-56cc45da-3289-4633-b77b-d2ccc852cf9c None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:30.095003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-56cc45da-3289-4633-b77b-d2ccc852cf9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:30.160659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 577/1155] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:30.184721 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69850690-aa2b-497c-9968-004ac87c1ac9 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:30.284691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69850690-aa2b-497c-9968-004ac87c1ac9 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:30.297805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:30.318835 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b7fc8-1180-4306-8999-32b33e474cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:30.319915 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b7fc8-1180-4306-8999-32b33e474cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:30.320149 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b7fc8-1180-4306-8999-32b33e474cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:30.320426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b7fc8-1180-4306-8999-32b33e474cd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:30.330296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b7fc8-1180-4306-8999-32b33e474cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:01:30.338026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-639b7fc8-1180-4306-8999-32b33e474cd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:01:30.338026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:30.338026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:30.338026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:30.338026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:30.338026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:30.338026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:30.338026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:30.338026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:30.338026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:30.338026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:30.338026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:30.338026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:01:30.338026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:01:30.338026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:01:30.338026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:01:30.339052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:01:30.339052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:01:30.339052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:01:30.339052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:01:30.339052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:30.339052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:30.339052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:30.339052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:30.339052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:30.339052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:30.339052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:30.339052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:30.339052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:30.339052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:30.339052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:30.339052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:30.340000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:30.340000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:30.340000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:30.340000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:30.340000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:01:30.340000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:01:30.340000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:01:30.340000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:01:30.340000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:01:30.340000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:01:30.340000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:01:30.340000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:01:30.340000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 579/1156] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:01:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:01:30.345159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc45da-3289-4633-b77b-d2ccc852cf9c None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:30.399301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:30.442458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc45da-3289-4633-b77b-d2ccc852cf9c None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:30.475161 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1667cdf-0417-40ec-86dc-0ce840717c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:30.475814 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1667cdf-0417-40ec-86dc-0ce840717c1d None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:30.476093 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1667cdf-0417-40ec-86dc-0ce840717c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:30.476367 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1667cdf-0417-40ec-86dc-0ce840717c1d None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:30.477108 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1667cdf-0417-40ec-86dc-0ce840717c1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:30.477413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1667cdf-0417-40ec-86dc-0ce840717c1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:30.478787 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1667cdf-0417-40ec-86dc-0ce840717c1d None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:30.487211 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 578/1157] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:01:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:30.492865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b7fc8-1180-4306-8999-32b33e474cd1 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:30.594307 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b7fc8-1180-4306-8999-32b33e474cd1 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:30.628324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2a357-95ca-446b-8b0b-90665bafaa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:30.629188 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2a357-95ca-446b-8b0b-90665bafaa74 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:30.629469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2a357-95ca-446b-8b0b-90665bafaa74 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:30.629744 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2a357-95ca-446b-8b0b-90665bafaa74 None admin] PATH_INFO: `/v3/users/0557a34b19c343da85e41dd0d8b4c1ff` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:30.630940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2a357-95ca-446b-8b0b-90665bafaa74 None admin] RBAC: Authorizing `identity:get_user(user_id=0557a34b19c343da85e41dd0d8b4c1ff)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:30.632366 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2a357-95ca-446b-8b0b-90665bafaa74 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:30.633963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 580/1158] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 09:01:30 2022] GET /identity/v3/users/0557a34b19c343da85e41dd0d8b4c1ff => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:30.640043 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1667cdf-0417-40ec-86dc-0ce840717c1d None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:30.737931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1667cdf-0417-40ec-86dc-0ce840717c1d None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:30.753951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:30.770847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24940a9-0ca5-4c82-8e59-e4faa6c85e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:30.771493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24940a9-0ca5-4c82-8e59-e4faa6c85e27 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:30.771774 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24940a9-0ca5-4c82-8e59-e4faa6c85e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:30.772047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24940a9-0ca5-4c82-8e59-e4faa6c85e27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:30.837221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24940a9-0ca5-4c82-8e59-e4faa6c85e27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:30.838649 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24940a9-0ca5-4c82-8e59-e4faa6c85e27 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:30.932597 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 579/1159] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:01:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 09:01:30.938648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2a357-95ca-446b-8b0b-90665bafaa74 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:30.979503 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:31.036749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2a357-95ca-446b-8b0b-90665bafaa74 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:31.068794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f92d2-bf92-428b-a963-72cd0e5f60bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:31.069895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f92d2-bf92-428b-a963-72cd0e5f60bd None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:31.070424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f92d2-bf92-428b-a963-72cd0e5f60bd None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:31.070424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f92d2-bf92-428b-a963-72cd0e5f60bd None admin] PATH_INFO: `/v3/projects/b2b390833758448288f09da36a6822c8` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:31.071631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f92d2-bf92-428b-a963-72cd0e5f60bd None admin] RBAC: Authorizing `identity:get_project(project_id=b2b390833758448288f09da36a6822c8)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:31.073058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f92d2-bf92-428b-a963-72cd0e5f60bd None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:31.075422 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 581/1160] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 09:01:30 2022] GET /identity/v3/projects/b2b390833758448288f09da36a6822c8 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:31.081394 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24940a9-0ca5-4c82-8e59-e4faa6c85e27 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:31.099488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:31.177185 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24940a9-0ca5-4c82-8e59-e4faa6c85e27 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:31.210750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274037f4-993f-4034-a472-128ad40ec8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:31.211346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274037f4-993f-4034-a472-128ad40ec8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:31.211628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274037f4-993f-4034-a472-128ad40ec8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:31.211905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274037f4-993f-4034-a472-128ad40ec8e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:31.212731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274037f4-993f-4034-a472-128ad40ec8e9 None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=b2b390833758448288f09da36a6822c8, user.id=0557a34b19c343da85e41dd0d8b4c1ff) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:31.213042 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274037f4-993f-4034-a472-128ad40ec8e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:31.214446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274037f4-993f-4034-a472-128ad40ec8e9 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:31.220827 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 580/1161] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:01:31 2022] GET /identity/v3/role_assignments?user.id=0557a34b19c343da85e41dd0d8b4c1ff&scope.project.id=b2b390833758448288f09da36a6822c8&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:33.255424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cede467-a253-45be-bd1c-0fed848bc06a None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:33.255979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cede467-a253-45be-bd1c-0fed848bc06a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:33.255979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cede467-a253-45be-bd1c-0fed848bc06a None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:33.257362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 582/1162] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:33.263072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a498-932f-45e2-848d-8d63f146daac None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:33.263492 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a498-932f-45e2-848d-8d63f146daac None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:33.263624 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a498-932f-45e2-848d-8d63f146daac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:33.280722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-7b51a498-932f-45e2-848d-8d63f146daac None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:33.281883 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7b51a498-932f-45e2-848d-8d63f146daac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:33.346586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 581/1163] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:33.368284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cede467-a253-45be-bd1c-0fed848bc06a None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:33.466557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cede467-a253-45be-bd1c-0fed848bc06a None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:33.475271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:33.500218 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1e68e-5fe5-4790-b61a-5afbfb2a3e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:33.500848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1e68e-5fe5-4790-b61a-5afbfb2a3e2d None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:33.501128 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1e68e-5fe5-4790-b61a-5afbfb2a3e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:33.501402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1e68e-5fe5-4790-b61a-5afbfb2a3e2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:33.566143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1e68e-5fe5-4790-b61a-5afbfb2a3e2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:33.567612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1e68e-5fe5-4790-b61a-5afbfb2a3e2d None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:33.661392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 583/1164] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:01:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:01:33.667288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51a498-932f-45e2-848d-8d63f146daac None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:33.681968 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:33.764966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51a498-932f-45e2-848d-8d63f146daac None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:33.799119 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a2112-bdac-434d-b70d-3898418e58b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:33.799921 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a2112-bdac-434d-b70d-3898418e58b7 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:33.800203 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a2112-bdac-434d-b70d-3898418e58b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:33.800484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a2112-bdac-434d-b70d-3898418e58b7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:33.807257 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a2112-bdac-434d-b70d-3898418e58b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:33.808659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a2112-bdac-434d-b70d-3898418e58b7 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-8b1a2112-bdac-434d-b70d-3898418e58b7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:33.822819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:33.824031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:33.824031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:33.824031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:33.824031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:33.824031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:33.824031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:33.824031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:01:33.824031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:01:33.824031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:01:33.824031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:01:33.824031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:33.824031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:33.824031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:33.824031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:33.824031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:33.824031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:33.825080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:33.825080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:33.825080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:33.825080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:33.825080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:33.825080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:33.825080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:33.825080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:33.825080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:33.825080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:33.825080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 20 09:01:33.825080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:01:33.825080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 09:01:33.825080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:01:33.825080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 582/1165] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jul 20 09:01:33 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:01:33.829913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1e68e-5fe5-4790-b61a-5afbfb2a3e2d None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:33.840397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:33.926315 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1e68e-5fe5-4790-b61a-5afbfb2a3e2d None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:33.958988 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186dbab-c2ab-4098-bf04-c6e085f228aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:33.959632 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186dbab-c2ab-4098-bf04-c6e085f228aa None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:33.959909 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186dbab-c2ab-4098-bf04-c6e085f228aa None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:33.960182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186dbab-c2ab-4098-bf04-c6e085f228aa None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:33.960888 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186dbab-c2ab-4098-bf04-c6e085f228aa None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:33.961192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186dbab-c2ab-4098-bf04-c6e085f228aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:33.962534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186dbab-c2ab-4098-bf04-c6e085f228aa None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:33.969921 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 584/1166] 10.222.0.44 () {56 vars in 1115 bytes} [Wed Jul 20 09:01:33 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:35.960364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ffae1-5f00-4fa6-8ce1-61ed6e7e32bc None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:35.960889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ffae1-5f00-4fa6-8ce1-61ed6e7e32bc None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:35.961002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ffae1-5f00-4fa6-8ce1-61ed6e7e32bc None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:35.962313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 583/1167] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:35.968174 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e26fc-101b-4e14-8c67-9f84abe23f94 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:35.968568 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e26fc-101b-4e14-8c67-9f84abe23f94 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:35.968734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e26fc-101b-4e14-8c67-9f84abe23f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:35.985864 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-db2e26fc-101b-4e14-8c67-9f84abe23f94 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:35.987082 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-db2e26fc-101b-4e14-8c67-9f84abe23f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:36.051046 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 585/1168] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:36.073375 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ffae1-5f00-4fa6-8ce1-61ed6e7e32bc None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:36.144231 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:36.174756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ffae1-5f00-4fa6-8ce1-61ed6e7e32bc None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:36.207083 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafb9e8-69f2-4def-a733-db5778a16de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:36.207719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafb9e8-69f2-4def-a733-db5778a16de9 None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:36.207996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafb9e8-69f2-4def-a733-db5778a16de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:36.208270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafb9e8-69f2-4def-a733-db5778a16de9 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:36.209008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dafb9e8-69f2-4def-a733-db5778a16de9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:36.210348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dafb9e8-69f2-4def-a733-db5778a16de9 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:36.224504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 584/1169] 10.222.0.44 () {60 vars in 1138 bytes} [Wed Jul 20 09:01:36 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:01:38.270361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffcbfd-8027-4848-b48f-310e8dabae30 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:38.270946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffcbfd-8027-4848-b48f-310e8dabae30 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:38.271057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffcbfd-8027-4848-b48f-310e8dabae30 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:38.272342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 586/1170] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:01:38.278080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cda1f9-e96e-46be-9b02-16f575251ad7 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:38.278470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cda1f9-e96e-46be-9b02-16f575251ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:38.278636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cda1f9-e96e-46be-9b02-16f575251ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:38.295593 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-03cda1f9-e96e-46be-9b02-16f575251ad7 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:38.296768 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-03cda1f9-e96e-46be-9b02-16f575251ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:38.362254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 585/1171] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:38.385070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffcbfd-8027-4848-b48f-310e8dabae30 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:38.387963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:38.486212 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffcbfd-8027-4848-b48f-310e8dabae30 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:38.519479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387673e-1d97-4c96-9d50-9d997e921f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:38.520329 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387673e-1d97-4c96-9d50-9d997e921f83 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:38.520669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387673e-1d97-4c96-9d50-9d997e921f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:38.520898 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387673e-1d97-4c96-9d50-9d997e921f83 None admin] PATH_INFO: `/v3/domains/e2f56d5e571e4f46b10484dbeb4b32c9` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:38.531855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387673e-1d97-4c96-9d50-9d997e921f83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e2f56d5e571e4f46b10484dbeb4b32c9)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:38.533323 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387673e-1d97-4c96-9d50-9d997e921f83 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:38.535025 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 587/1172] 10.222.0.44 () {56 vars in 1183 bytes} [Wed Jul 20 09:01:38 2022] GET /identity/v3/domains/e2f56d5e571e4f46b10484dbeb4b32c9 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:38.541013 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cda1f9-e96e-46be-9b02-16f575251ad7 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:38.599999 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:38.638892 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cda1f9-e96e-46be-9b02-16f575251ad7 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:38.672104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f6e5d-ddc8-4e92-9c51-bb665ccd673f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:38.672759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f6e5d-ddc8-4e92-9c51-bb665ccd673f None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:38.673044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f6e5d-ddc8-4e92-9c51-bb665ccd673f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:38.673324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f6e5d-ddc8-4e92-9c51-bb665ccd673f None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:38.674071 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f6e5d-ddc8-4e92-9c51-bb665ccd673f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:38.675526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f6e5d-ddc8-4e92-9c51-bb665ccd673f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:38.700123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 586/1173] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jul 20 09:01:38 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:01:40.772429 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca901c3-1143-42b9-88ae-3bc124039d77 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:40.772989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca901c3-1143-42b9-88ae-3bc124039d77 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:40.772989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca901c3-1143-42b9-88ae-3bc124039d77 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:40.774352 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 588/1174] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:40.779540 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b4abb-e922-4d91-9ca7-5a38e016f2e8 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:40.779915 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b4abb-e922-4d91-9ca7-5a38e016f2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:40.780101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b4abb-e922-4d91-9ca7-5a38e016f2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:40.797000 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-ce6b4abb-e922-4d91-9ca7-5a38e016f2e8 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:40.798158 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ce6b4abb-e922-4d91-9ca7-5a38e016f2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:40.865349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 587/1175] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:40.891430 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca901c3-1143-42b9-88ae-3bc124039d77 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:40.953971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:40.993262 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca901c3-1143-42b9-88ae-3bc124039d77 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:41.027845 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db67e16-258e-4953-bf80-206098f64438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:41.028698 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db67e16-258e-4953-bf80-206098f64438 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:41.028980 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db67e16-258e-4953-bf80-206098f64438 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:41.029254 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db67e16-258e-4953-bf80-206098f64438 None admin] PATH_INFO: `/v3/domains/e2f56d5e571e4f46b10484dbeb4b32c9` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:41.030409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db67e16-258e-4953-bf80-206098f64438 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e2f56d5e571e4f46b10484dbeb4b32c9)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:41.031855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db67e16-258e-4953-bf80-206098f64438 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:41.033425 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 589/1176] 10.222.0.44 () {56 vars in 1183 bytes} [Wed Jul 20 09:01:40 2022] GET /identity/v3/domains/e2f56d5e571e4f46b10484dbeb4b32c9 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:41.039149 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b4abb-e922-4d91-9ca7-5a38e016f2e8 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:41.083957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:41.139132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b4abb-e922-4d91-9ca7-5a38e016f2e8 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:41.173931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05ffce-5bb1-47dc-ae82-e672e7d366df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:41.174603 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05ffce-5bb1-47dc-ae82-e672e7d366df None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:41.174915 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05ffce-5bb1-47dc-ae82-e672e7d366df None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:41.175211 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05ffce-5bb1-47dc-ae82-e672e7d366df None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:41.176291 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05ffce-5bb1-47dc-ae82-e672e7d366df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:41.177724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05ffce-5bb1-47dc-ae82-e672e7d366df None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:41.197469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 588/1177] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 09:01:41 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:01:43.249562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8642ca9-d4a7-4281-b97c-794e350ad5ce None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:43.250121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8642ca9-d4a7-4281-b97c-794e350ad5ce None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:43.250121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8642ca9-d4a7-4281-b97c-794e350ad5ce None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:43.251519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 590/1178] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:43.256751 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5221e60-a6bc-4987-b58f-1eecbea1272e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:43.257161 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5221e60-a6bc-4987-b58f-1eecbea1272e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:43.257324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5221e60-a6bc-4987-b58f-1eecbea1272e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:43.274608 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-e5221e60-a6bc-4987-b58f-1eecbea1272e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:43.275809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e5221e60-a6bc-4987-b58f-1eecbea1272e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:43.341180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 589/1179] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:43.364342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8642ca9-d4a7-4281-b97c-794e350ad5ce None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:43.463594 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8642ca9-d4a7-4281-b97c-794e350ad5ce None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:43.490522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:43.497135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f20ef8-322a-45e8-9218-efc17624b801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:43.497914 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f20ef8-322a-45e8-9218-efc17624b801 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:43.498197 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f20ef8-322a-45e8-9218-efc17624b801 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:43.498474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f20ef8-322a-45e8-9218-efc17624b801 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:43.508280 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f20ef8-322a-45e8-9218-efc17624b801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:01:43.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-66f20ef8-322a-45e8-9218-efc17624b801 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:01:43.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:43.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:43.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:43.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:43.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:43.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:43.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:43.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:43.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:43.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:43.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:43.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:01:43.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:01:43.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:01:43.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:01:43.517679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:01:43.517679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:01:43.517679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:01:43.517679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:01:43.517679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:43.517679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:43.517679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:43.517679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:43.517679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:43.517679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:43.517679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:43.517679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:43.517679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:43.517679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:43.517679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:43.517679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:43.518961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:43.518961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:43.518961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:43.518961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:43.518961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:01:43.518961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:01:43.518961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:01:43.518961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:01:43.518961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:01:43.518961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:01:43.518961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:01:43.518961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:01:43.518961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 591/1180] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:01:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:01:43.523541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5221e60-a6bc-4987-b58f-1eecbea1272e None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:43.552133 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:43.620458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5221e60-a6bc-4987-b58f-1eecbea1272e None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:43.653991 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680dde7-33d1-4865-a35b-8a82afbd3d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:43.654651 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680dde7-33d1-4865-a35b-8a82afbd3d8d None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:43.654967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680dde7-33d1-4865-a35b-8a82afbd3d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:43.655246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680dde7-33d1-4865-a35b-8a82afbd3d8d None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:43.655985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680dde7-33d1-4865-a35b-8a82afbd3d8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:43.656293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680dde7-33d1-4865-a35b-8a82afbd3d8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:43.657648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680dde7-33d1-4865-a35b-8a82afbd3d8d None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:43.665467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 590/1181] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:01:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:43.671268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f20ef8-322a-45e8-9218-efc17624b801 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:43.769553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f20ef8-322a-45e8-9218-efc17624b801 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:43.801796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fb810-e452-4d28-99e5-d888ce4f5b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:43.802651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fb810-e452-4d28-99e5-d888ce4f5b03 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:43.802955 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fb810-e452-4d28-99e5-d888ce4f5b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:43.803238 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fb810-e452-4d28-99e5-d888ce4f5b03 None admin] PATH_INFO: `/v3/users/84677f446f2d4b1c9b2d84f523ce7b37` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:43.816197 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5fb810-e452-4d28-99e5-d888ce4f5b03 None admin] RBAC: Authorizing `identity:get_user(user_id=84677f446f2d4b1c9b2d84f523ce7b37)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:43.817621 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5fb810-e452-4d28-99e5-d888ce4f5b03 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:43.819261 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 592/1182] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 09:01:43 2022] GET /identity/v3/users/84677f446f2d4b1c9b2d84f523ce7b37 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:43.826177 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680dde7-33d1-4865-a35b-8a82afbd3d8d None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:43.914540 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:43.924459 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680dde7-33d1-4865-a35b-8a82afbd3d8d None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:43.957792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfae80-3d2e-4fef-99c5-c18e93c48c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:43.958446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfae80-3d2e-4fef-99c5-c18e93c48c2a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:43.958758 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfae80-3d2e-4fef-99c5-c18e93c48c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:43.959040 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfae80-3d2e-4fef-99c5-c18e93c48c2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:44.023846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfae80-3d2e-4fef-99c5-c18e93c48c2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:44.025235 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfae80-3d2e-4fef-99c5-c18e93c48c2a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:44.111914 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:44.121927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 591/1183] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:01:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:01:44.127867 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fb810-e452-4d28-99e5-d888ce4f5b03 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:44.175914 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:44.225223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fb810-e452-4d28-99e5-d888ce4f5b03 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:44.258640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a34f32-8625-46fc-a641-86ae70d34314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:44.259524 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a34f32-8625-46fc-a641-86ae70d34314 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:44.259805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a34f32-8625-46fc-a641-86ae70d34314 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:44.260088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a34f32-8625-46fc-a641-86ae70d34314 None admin] PATH_INFO: `/v3/projects/0f24a147ee104a1c83f00786905c8baf` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:44.261246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a34f32-8625-46fc-a641-86ae70d34314 None admin] RBAC: Authorizing `identity:get_project(project_id=0f24a147ee104a1c83f00786905c8baf)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:44.263368 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a34f32-8625-46fc-a641-86ae70d34314 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:44.265027 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 593/1184] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 09:01:44 2022] GET /identity/v3/projects/0f24a147ee104a1c83f00786905c8baf => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:44.271046 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfae80-3d2e-4fef-99c5-c18e93c48c2a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:44.367079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfae80-3d2e-4fef-99c5-c18e93c48c2a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:44.400385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb1efb-75f6-4114-9d4c-688070f23db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:44.401029 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb1efb-75f6-4114-9d4c-688070f23db7 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:44.401314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb1efb-75f6-4114-9d4c-688070f23db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:44.401591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb1efb-75f6-4114-9d4c-688070f23db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:44.402496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb1efb-75f6-4114-9d4c-688070f23db7 None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=0f24a147ee104a1c83f00786905c8baf, user.id=84677f446f2d4b1c9b2d84f523ce7b37) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:44.402876 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb1efb-75f6-4114-9d4c-688070f23db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:44.404300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb1efb-75f6-4114-9d4c-688070f23db7 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:44.409591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 592/1185] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:01:44 2022] GET /identity/v3/role_assignments?user.id=84677f446f2d4b1c9b2d84f523ce7b37&scope.project.id=0f24a147ee104a1c83f00786905c8baf&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:46.432833 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972ccd6-6883-48cb-bb5a-af6b64c9e8ef None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:46.433362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972ccd6-6883-48cb-bb5a-af6b64c9e8ef None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:46.433562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3972ccd6-6883-48cb-bb5a-af6b64c9e8ef None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:46.435230 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 594/1186] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:46.440380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7071b3-e0a3-43a9-8f9e-afbbba6a60e2 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:46.440801 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7071b3-e0a3-43a9-8f9e-afbbba6a60e2 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:46.440930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7071b3-e0a3-43a9-8f9e-afbbba6a60e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:46.457273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-5a7071b3-e0a3-43a9-8f9e-afbbba6a60e2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:46.458438 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5a7071b3-e0a3-43a9-8f9e-afbbba6a60e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:46.507819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:46.524041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 593/1187] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:46.547431 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972ccd6-6883-48cb-bb5a-af6b64c9e8ef None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:46.647180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3972ccd6-6883-48cb-bb5a-af6b64c9e8ef None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:46.663073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:46.680934 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c664e-45c3-4d65-a6b6-04c106fa036a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:46.681714 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c664e-45c3-4d65-a6b6-04c106fa036a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:46.681997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c664e-45c3-4d65-a6b6-04c106fa036a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:46.682271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c664e-45c3-4d65-a6b6-04c106fa036a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:46.692569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c664e-45c3-4d65-a6b6-04c106fa036a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:01:46.701036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-e07c664e-45c3-4d65-a6b6-04c106fa036a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:01:46.701036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:46.701036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:46.701036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:46.701036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:46.701036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:46.701036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:46.701036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:46.701036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:46.701036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:46.701036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:46.701036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:46.701036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:01:46.701036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:01:46.701036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:01:46.701036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:01:46.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:01:46.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:01:46.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:01:46.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:01:46.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:46.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:46.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:46.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:46.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:46.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:46.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:46.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:46.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:46.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:46.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:46.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:46.703087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:46.703087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:46.703087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:46.703087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:46.703087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:01:46.703087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:01:46.703087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:01:46.703087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:01:46.703087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:01:46.703087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:01:46.703087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:01:46.703087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:01:46.703087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 595/1188] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:01:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:01:46.708262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7071b3-e0a3-43a9-8f9e-afbbba6a60e2 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:46.806840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7071b3-e0a3-43a9-8f9e-afbbba6a60e2 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:46.840456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf6da8-418b-499b-9507-de05ec6d7be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:46.841123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf6da8-418b-499b-9507-de05ec6d7be7 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:46.841412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf6da8-418b-499b-9507-de05ec6d7be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:46.841696 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf6da8-418b-499b-9507-de05ec6d7be7 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:46.842444 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf6da8-418b-499b-9507-de05ec6d7be7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:46.842786 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf6da8-418b-499b-9507-de05ec6d7be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:46.844145 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf6da8-418b-499b-9507-de05ec6d7be7 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:46.852118 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 594/1189] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:01:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:46.858595 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c664e-45c3-4d65-a6b6-04c106fa036a None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:46.955014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c664e-45c3-4d65-a6b6-04c106fa036a None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:46.987303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63164bc3-7e5d-419d-bed1-fa0f79f4ee68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:46.988148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63164bc3-7e5d-419d-bed1-fa0f79f4ee68 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:46.988422 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63164bc3-7e5d-419d-bed1-fa0f79f4ee68 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:46.988691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63164bc3-7e5d-419d-bed1-fa0f79f4ee68 None admin] PATH_INFO: `/v3/users/84677f446f2d4b1c9b2d84f523ce7b37` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:46.989857 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63164bc3-7e5d-419d-bed1-fa0f79f4ee68 None admin] RBAC: Authorizing `identity:get_user(user_id=84677f446f2d4b1c9b2d84f523ce7b37)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:46.991311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63164bc3-7e5d-419d-bed1-fa0f79f4ee68 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:46.992887 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 596/1190] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 09:01:46 2022] GET /identity/v3/users/84677f446f2d4b1c9b2d84f523ce7b37 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:46.999704 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf6da8-418b-499b-9507-de05ec6d7be7 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:47.048163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:47.098206 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf6da8-418b-499b-9507-de05ec6d7be7 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:47.131699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd788769-8ecb-431b-bf9c-7fafde9a1b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:47.132544 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd788769-8ecb-431b-bf9c-7fafde9a1b33 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:47.132830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd788769-8ecb-431b-bf9c-7fafde9a1b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:47.133117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd788769-8ecb-431b-bf9c-7fafde9a1b33 None admin] PATH_INFO: `/v3/projects/0f24a147ee104a1c83f00786905c8baf` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:47.134283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd788769-8ecb-431b-bf9c-7fafde9a1b33 None admin] RBAC: Authorizing `identity:get_project(project_id=0f24a147ee104a1c83f00786905c8baf)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:47.135752 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd788769-8ecb-431b-bf9c-7fafde9a1b33 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:47.137479 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 595/1191] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 09:01:46 2022] GET /identity/v3/projects/0f24a147ee104a1c83f00786905c8baf => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:47.143525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63164bc3-7e5d-419d-bed1-fa0f79f4ee68 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:47.188474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:47.242124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63164bc3-7e5d-419d-bed1-fa0f79f4ee68 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:47.274196 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240fc41-be42-4eb0-82fe-f05961329985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:47.275119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240fc41-be42-4eb0-82fe-f05961329985 None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:47.275400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240fc41-be42-4eb0-82fe-f05961329985 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:47.275677 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240fc41-be42-4eb0-82fe-f05961329985 None admin] PATH_INFO: `/v3/projects/0f24a147ee104a1c83f00786905c8baf/users/84677f446f2d4b1c9b2d84f523ce7b37/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:47.278145 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240fc41-be42-4eb0-82fe-f05961329985 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f24a147ee104a1c83f00786905c8baf, user_id=84677f446f2d4b1c9b2d84f523ce7b37, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:47.279597 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240fc41-be42-4eb0-82fe-f05961329985 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:47.291561 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 597/1192] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 09:01:47 2022] PUT /identity/v3/projects/0f24a147ee104a1c83f00786905c8baf/users/84677f446f2d4b1c9b2d84f523ce7b37/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:01:49.280917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985df8b0-f3c8-44a8-9a37-b21007f377d0 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:49.281549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985df8b0-f3c8-44a8-9a37-b21007f377d0 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:49.281549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985df8b0-f3c8-44a8-9a37-b21007f377d0 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:49.282910 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 596/1193] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:01:49.288122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04aac6d-e9e6-4bc6-a27d-957481a9429d None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:49.288535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04aac6d-e9e6-4bc6-a27d-957481a9429d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:49.288677 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04aac6d-e9e6-4bc6-a27d-957481a9429d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:49.305017 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-f04aac6d-e9e6-4bc6-a27d-957481a9429d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:01:49.306194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f04aac6d-e9e6-4bc6-a27d-957481a9429d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:01:49.371232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 598/1194] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:01:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:01:49.395087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985df8b0-f3c8-44a8-9a37-b21007f377d0 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:49.494867 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985df8b0-f3c8-44a8-9a37-b21007f377d0 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:49.509958 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:49.527700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87231b3e-7251-474e-b55e-0631247657c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:49.528477 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87231b3e-7251-474e-b55e-0631247657c6 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:49.528757 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87231b3e-7251-474e-b55e-0631247657c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:49.529031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87231b3e-7251-474e-b55e-0631247657c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:49.539382 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87231b3e-7251-474e-b55e-0631247657c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:01:49.546997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-87231b3e-7251-474e-b55e-0631247657c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:01:49.546997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:01:49.546997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:01:49.546997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:01:49.546997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:01:49.546997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:01:49.546997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:01:49.546997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:01:49.546997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:01:49.546997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:01:49.546997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:01:49.546997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:01:49.546997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:01:49.546997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:01:49.546997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:01:49.546997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:01:49.548137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:01:49.548137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:01:49.548137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:01:49.548137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:01:49.548137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:01:49.548137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:01:49.548137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:01:49.548137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:01:49.548137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:01:49.548137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:01:49.548137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:01:49.548137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:01:49.548137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:01:49.548137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:01:49.548137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:01:49.548137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:01:49.549117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:01:49.549117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:01:49.549117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:01:49.549117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:01:49.549117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:01:49.549117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:01:49.549117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:01:49.549117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:01:49.549117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:01:49.549117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:01:49.549117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:01:49.549117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:01:49.549117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 597/1195] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:01:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:01:49.555510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04aac6d-e9e6-4bc6-a27d-957481a9429d None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:49.589611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:49.654114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04aac6d-e9e6-4bc6-a27d-957481a9429d None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:49.686885 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1dd75-b959-4141-a203-71630bff377b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:49.687548 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1dd75-b959-4141-a203-71630bff377b None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:49.687836 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1dd75-b959-4141-a203-71630bff377b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:49.688119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1dd75-b959-4141-a203-71630bff377b None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:49.688864 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1dd75-b959-4141-a203-71630bff377b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:49.689174 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1dd75-b959-4141-a203-71630bff377b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:49.690543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b1dd75-b959-4141-a203-71630bff377b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:49.698631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 599/1196] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:01:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:49.704952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87231b3e-7251-474e-b55e-0631247657c6 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:49.801805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87231b3e-7251-474e-b55e-0631247657c6 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:49.834556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b3677-77e2-4c1b-9214-7b0dae6de1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:49.835443 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b3677-77e2-4c1b-9214-7b0dae6de1ea None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:49.835727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b3677-77e2-4c1b-9214-7b0dae6de1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:49.836003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b3677-77e2-4c1b-9214-7b0dae6de1ea None admin] PATH_INFO: `/v3/users/84677f446f2d4b1c9b2d84f523ce7b37` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:49.837168 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b3677-77e2-4c1b-9214-7b0dae6de1ea None admin] RBAC: Authorizing `identity:get_user(user_id=84677f446f2d4b1c9b2d84f523ce7b37)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:49.838598 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571b3677-77e2-4c1b-9214-7b0dae6de1ea None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:49.840263 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 598/1197] 10.222.0.44 () {56 vars in 1177 bytes} [Wed Jul 20 09:01:49 2022] GET /identity/v3/users/84677f446f2d4b1c9b2d84f523ce7b37 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:49.846013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1dd75-b959-4141-a203-71630bff377b None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:49.939071 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:49.945098 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1dd75-b959-4141-a203-71630bff377b None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:49.977058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f07965-0e4c-4bc6-a603-8f53845873d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:49.977692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f07965-0e4c-4bc6-a603-8f53845873d9 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:49.977971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f07965-0e4c-4bc6-a603-8f53845873d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:49.978244 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f07965-0e4c-4bc6-a603-8f53845873d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:50.043101 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f07965-0e4c-4bc6-a603-8f53845873d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:50.044519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f07965-0e4c-4bc6-a603-8f53845873d9 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:50.133317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:50.139890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 600/1198] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:01:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:01:50.145759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b3677-77e2-4c1b-9214-7b0dae6de1ea None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:50.190515 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:01:50.244301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b3677-77e2-4c1b-9214-7b0dae6de1ea None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:50.277559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19715398-62fd-45fc-9368-9ae42e618597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:50.278412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19715398-62fd-45fc-9368-9ae42e618597 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:50.278710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19715398-62fd-45fc-9368-9ae42e618597 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:50.278991 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19715398-62fd-45fc-9368-9ae42e618597 None admin] PATH_INFO: `/v3/projects/0f24a147ee104a1c83f00786905c8baf` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:50.280152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19715398-62fd-45fc-9368-9ae42e618597 None admin] RBAC: Authorizing `identity:get_project(project_id=0f24a147ee104a1c83f00786905c8baf)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:50.281573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19715398-62fd-45fc-9368-9ae42e618597 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:50.283301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 599/1199] 10.222.0.44 () {56 vars in 1186 bytes} [Wed Jul 20 09:01:50 2022] GET /identity/v3/projects/0f24a147ee104a1c83f00786905c8baf => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:01:50.289362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f07965-0e4c-4bc6-a603-8f53845873d9 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:01:50.385353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f07965-0e4c-4bc6-a603-8f53845873d9 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:01:50.419161 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee9a1b-3bb7-44a6-8fbd-8fa9fa75002f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:01:50.419804 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee9a1b-3bb7-44a6-8fbd-8fa9fa75002f None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:01:50.420090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee9a1b-3bb7-44a6-8fbd-8fa9fa75002f None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:01:50.420371 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee9a1b-3bb7-44a6-8fbd-8fa9fa75002f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:01:50.421193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee9a1b-3bb7-44a6-8fbd-8fa9fa75002f None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=0f24a147ee104a1c83f00786905c8baf, user.id=84677f446f2d4b1c9b2d84f523ce7b37) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:01:50.421505 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee9a1b-3bb7-44a6-8fbd-8fa9fa75002f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:01:50.422946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee9a1b-3bb7-44a6-8fbd-8fa9fa75002f None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:01:50.428596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 601/1200] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:01:50 2022] GET /identity/v3/role_assignments?user.id=84677f446f2d4b1c9b2d84f523ce7b37&scope.project.id=0f24a147ee104a1c83f00786905c8baf&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:02:00.272730 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=44 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:00.425991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=70 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:45.906462 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbd21c-03a5-41eb-ab24-37cfeb58491c None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:45.907098 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbd21c-03a5-41eb-ab24-37cfeb58491c None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:45.907098 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebbd21c-03a5-41eb-ab24-37cfeb58491c None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:45.908480 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 600/1201] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:02:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:02:45.914461 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c6b07-0056-4ad7-8002-2373bd39c94a None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:45.914928 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c6b07-0056-4ad7-8002-2373bd39c94a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:45.915335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c6b07-0056-4ad7-8002-2373bd39c94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:45.936894 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-990c6b07-0056-4ad7-8002-2373bd39c94a None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:02:45.938120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-990c6b07-0056-4ad7-8002-2373bd39c94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:02:46.042884 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 602/1202] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:02:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:02:46.072863 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbd21c-03a5-41eb-ab24-37cfeb58491c None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:46.218836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebbd21c-03a5-41eb-ab24-37cfeb58491c None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:46.277969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f049d-09ba-4707-a6f1-0aaa16d9d888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:46.278874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f049d-09ba-4707-a6f1-0aaa16d9d888 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:46.279251 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f049d-09ba-4707-a6f1-0aaa16d9d888 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:46.279641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f049d-09ba-4707-a6f1-0aaa16d9d888 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:46.298433 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f049d-09ba-4707-a6f1-0aaa16d9d888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:46.300414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f049d-09ba-4707-a6f1-0aaa16d9d888 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:46.324434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-652f049d-09ba-4707-a6f1-0aaa16d9d888 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:02:46.324434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:02:46.324434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:02:46.324434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:02:46.324434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:02:46.324434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:02:46.324434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:02:46.324434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:02:46.324434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:02:46.324434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:02:46.324434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:02:46.324434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:02:46.324434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:02:46.324434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:02:46.324434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:02:46.325604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:02:46.326586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:02:46.326586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:02:46.326586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:02:46.326586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:02:46.326586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 601/1203] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:02:46.334327 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c6b07-0056-4ad7-8002-2373bd39c94a None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:46.473404 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c6b07-0056-4ad7-8002-2373bd39c94a None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:46.508915 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:46.516789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae13c4-54ab-43f6-b713-b35ed27cac62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:46.517535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae13c4-54ab-43f6-b713-b35ed27cac62 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:46.517889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae13c4-54ab-43f6-b713-b35ed27cac62 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:46.518171 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae13c4-54ab-43f6-b713-b35ed27cac62 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:46.518947 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae13c4-54ab-43f6-b713-b35ed27cac62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:02:46.519246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae13c4-54ab-43f6-b713-b35ed27cac62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:46.520623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae13c4-54ab-43f6-b713-b35ed27cac62 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:46.535184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 603/1204] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:02:46.541289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f049d-09ba-4707-a6f1-0aaa16d9d888 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:46.616623 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:46.669687 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f049d-09ba-4707-a6f1-0aaa16d9d888 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:46.711344 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02554eef-f525-497c-b45c-1c77017eb7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:46.712155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02554eef-f525-497c-b45c-1c77017eb7da None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:46.712564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02554eef-f525-497c-b45c-1c77017eb7da None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:46.712977 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02554eef-f525-497c-b45c-1c77017eb7da None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:46.713853 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02554eef-f525-497c-b45c-1c77017eb7da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:46.715456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02554eef-f525-497c-b45c-1c77017eb7da None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:46.746803 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 602/1205] 10.222.0.44 () {60 vars in 1132 bytes} [Wed Jul 20 09:02:46 2022] POST /identity/v3/users => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:02:48.906476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef41e02-2855-491a-a82e-af3e30959737 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:48.907056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef41e02-2855-491a-a82e-af3e30959737 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:48.907159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef41e02-2855-491a-a82e-af3e30959737 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:48.908964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 604/1206] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:02:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:02:48.914217 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8e296-4d4c-4c38-a7fb-cdd7ba84a9cb None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:48.914776 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8e296-4d4c-4c38-a7fb-cdd7ba84a9cb None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:48.914876 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8e296-4d4c-4c38-a7fb-cdd7ba84a9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:48.934021 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-f8b8e296-4d4c-4c38-a7fb-cdd7ba84a9cb None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:02:48.935485 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f8b8e296-4d4c-4c38-a7fb-cdd7ba84a9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:02:49.014939 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 603/1207] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:02:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:02:49.037837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef41e02-2855-491a-a82e-af3e30959737 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:49.151619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef41e02-2855-491a-a82e-af3e30959737 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:49.195793 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b2835-3590-4388-beeb-ae3aba222d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:49.196540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b2835-3590-4388-beeb-ae3aba222d53 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:49.197462 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b2835-3590-4388-beeb-ae3aba222d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:49.197662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b2835-3590-4388-beeb-ae3aba222d53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:49.208743 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b2835-3590-4388-beeb-ae3aba222d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:02:49.218667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-471b2835-3590-4388-beeb-ae3aba222d53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:02:49.218667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:02:49.218667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:02:49.218667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:02:49.218667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:02:49.218667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:02:49.218667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:02:49.218667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:02:49.218667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:02:49.218667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:02:49.218667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:02:49.218667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:02:49.218667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:02:49.218667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:02:49.218667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:02:49.218667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:02:49.219682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:02:49.219682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:02:49.219682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:02:49.219682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:02:49.219682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:02:49.219682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:02:49.219682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:02:49.219682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:02:49.219682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:02:49.219682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:02:49.219682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:02:49.219682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:02:49.219682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:02:49.219682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:02:49.219682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:02:49.219682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:02:49.225616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:02:49.225616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:02:49.225616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:02:49.225616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:02:49.225616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:02:49.225616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:02:49.225616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:02:49.225616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:02:49.225616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:02:49.225616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:02:49.225616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:02:49.225616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:02:49.225616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 605/1208] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:02:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:02:49.226342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8e296-4d4c-4c38-a7fb-cdd7ba84a9cb None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:49.273104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:49.346853 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8e296-4d4c-4c38-a7fb-cdd7ba84a9cb None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:49.389536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae55d12-8ab6-4d53-bf5a-e53943254c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:49.390282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae55d12-8ab6-4d53-bf5a-e53943254c03 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:49.390548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae55d12-8ab6-4d53-bf5a-e53943254c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:49.390957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae55d12-8ab6-4d53-bf5a-e53943254c03 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:49.391649 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae55d12-8ab6-4d53-bf5a-e53943254c03 None admin] RBAC: Adding query filter params (name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:02:49.392020 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae55d12-8ab6-4d53-bf5a-e53943254c03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:49.393422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae55d12-8ab6-4d53-bf5a-e53943254c03 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:49.407771 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 604/1209] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 09:02:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:02:49.413514 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b2835-3590-4388-beeb-ae3aba222d53 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:49.475676 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:49.531532 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b2835-3590-4388-beeb-ae3aba222d53 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:49.567110 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbe4b3-b8c1-4ebc-8857-6256b27b0584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:49.567971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbe4b3-b8c1-4ebc-8857-6256b27b0584 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:49.568250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbe4b3-b8c1-4ebc-8857-6256b27b0584 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:49.568557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbe4b3-b8c1-4ebc-8857-6256b27b0584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:49.581030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbe4b3-b8c1-4ebc-8857-6256b27b0584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:49.582705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbe4b3-b8c1-4ebc-8857-6256b27b0584 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:49.600610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-57bbe4b3-b8c1-4ebc-8857-6256b27b0584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:02:49.600610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:02:49.600610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:02:49.600610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:02:49.600610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:02:49.600610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:02:49.600610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:02:49.600610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:02:49.600610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:02:49.600610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:02:49.600610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:02:49.600610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:02:49.600610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:02:49.600610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:02:49.600610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:02:49.601615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:02:49.602525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:02:49.602525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:02:49.602525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:02:49.602525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:02:49.608985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 606/1210] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:02:49.609152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae55d12-8ab6-4d53-bf5a-e53943254c03 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:49.718540 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:49.721653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae55d12-8ab6-4d53-bf5a-e53943254c03 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:49.760531 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9110f8-b7f7-4139-a54f-b0f5b7a8d2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:49.761488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9110f8-b7f7-4139-a54f-b0f5b7a8d2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:49.761776 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9110f8-b7f7-4139-a54f-b0f5b7a8d2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:49.762060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9110f8-b7f7-4139-a54f-b0f5b7a8d2a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:49.762840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9110f8-b7f7-4139-a54f-b0f5b7a8d2a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:02:49.763152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9110f8-b7f7-4139-a54f-b0f5b7a8d2a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:49.764525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9110f8-b7f7-4139-a54f-b0f5b7a8d2a4 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:49.778039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 605/1211] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:02:49.783869 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbe4b3-b8c1-4ebc-8857-6256b27b0584 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:49.909709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbe4b3-b8c1-4ebc-8857-6256b27b0584 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:49.911592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:49.951803 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cc19d-877e-4d81-93d1-fd32b958d5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:49.952654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cc19d-877e-4d81-93d1-fd32b958d5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:49.952978 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cc19d-877e-4d81-93d1-fd32b958d5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:49.953283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cc19d-877e-4d81-93d1-fd32b958d5e6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:49.961412 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cc19d-877e-4d81-93d1-fd32b958d5e6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:49.963625 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cc19d-877e-4d81-93d1-fd32b958d5e6 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:49.981920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-415cc19d-877e-4d81-93d1-fd32b958d5e6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:02:49.981920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:02:49.981920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:02:49.981920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:02:49.981920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:02:49.981920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:02:49.981920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:02:49.981920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:02:49.981920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:02:49.981920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:02:49.981920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:02:49.981920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:02:49.981920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:02:49.981920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:02:49.981920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:02:49.981920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:02:49.983166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:02:49.984163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:02:49.984163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:02:49.984163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:02:49.984163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:02:49.984163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:02:49.984163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:02:49.984163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:02:49.984163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:02:49.984163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:02:49.984163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:02:49.989947 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 607/1212] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:02:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:02:49.990080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9110f8-b7f7-4139-a54f-b0f5b7a8d2a4 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:50.104655 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9110f8-b7f7-4139-a54f-b0f5b7a8d2a4 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:50.145747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005634f-b099-4ff2-888c-862728f13cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:50.146452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005634f-b099-4ff2-888c-862728f13cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:50.146724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005634f-b099-4ff2-888c-862728f13cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:50.147014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005634f-b099-4ff2-888c-862728f13cf7 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:50.147851 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005634f-b099-4ff2-888c-862728f13cf7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:02:50.148158 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005634f-b099-4ff2-888c-862728f13cf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:50.149566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005634f-b099-4ff2-888c-862728f13cf7 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:50.174742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 606/1213] 10.222.0.44 () {56 vars in 1143 bytes} [Wed Jul 20 09:02:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:02:50.176630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cc19d-877e-4d81-93d1-fd32b958d5e6 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:50.304550 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cc19d-877e-4d81-93d1-fd32b958d5e6 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:50.345762 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bfbc7-a4d3-494c-892e-b7dd39ebdf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:50.346417 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bfbc7-a4d3-494c-892e-b7dd39ebdf8a None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:50.346714 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bfbc7-a4d3-494c-892e-b7dd39ebdf8a None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:50.346997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bfbc7-a4d3-494c-892e-b7dd39ebdf8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:50.383506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:50.429429 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111bfbc7-a4d3-494c-892e-b7dd39ebdf8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:50.430888 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111bfbc7-a4d3-494c-892e-b7dd39ebdf8a None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:50.567665 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 608/1214] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:02:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 09:02:50.573821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005634f-b099-4ff2-888c-862728f13cf7 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:50.596778 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:50.685596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005634f-b099-4ff2-888c-862728f13cf7 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:50.720567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8fc6e7-b280-445a-9e8f-5a38fef803a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:50.721361 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fc6e7-b280-445a-9e8f-5a38fef803a6 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:50.721641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fc6e7-b280-445a-9e8f-5a38fef803a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:50.721918 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8fc6e7-b280-445a-9e8f-5a38fef803a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:50.733378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8fc6e7-b280-445a-9e8f-5a38fef803a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:50.734824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8fc6e7-b280-445a-9e8f-5a38fef803a6 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:50.754491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-ee8fc6e7-b280-445a-9e8f-5a38fef803a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:02:50.754491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:02:50.754491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:02:50.754491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:02:50.754491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:02:50.754491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:02:50.754491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:02:50.754491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:02:50.754491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:02:50.754491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:02:50.754491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:02:50.754491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:02:50.754491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:02:50.754491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:02:50.754491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:02:50.755645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:02:50.756541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:02:50.756541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:02:50.756541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:02:50.756541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:02:50.764392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 607/1215] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:02:50.764567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bfbc7-a4d3-494c-892e-b7dd39ebdf8a None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:50.835522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:50.887022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bfbc7-a4d3-494c-892e-b7dd39ebdf8a None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:50.927935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f9430-b3f7-4a5d-8280-e4bc0a131cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:50.928692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f9430-b3f7-4a5d-8280-e4bc0a131cbe None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:50.928888 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f9430-b3f7-4a5d-8280-e4bc0a131cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:50.929150 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f9430-b3f7-4a5d-8280-e4bc0a131cbe None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:50.929870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f9430-b3f7-4a5d-8280-e4bc0a131cbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:02:50.930177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f9430-b3f7-4a5d-8280-e4bc0a131cbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:50.931733 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f9430-b3f7-4a5d-8280-e4bc0a131cbe None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:50.944362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 609/1216] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:02:50.951062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8fc6e7-b280-445a-9e8f-5a38fef803a6 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:51.042019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:51.080202 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8fc6e7-b280-445a-9e8f-5a38fef803a6 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:51.117023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060642e-674e-4603-8963-a097994f8eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:51.117824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060642e-674e-4603-8963-a097994f8eba None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:51.118110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060642e-674e-4603-8963-a097994f8eba None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:51.118399 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060642e-674e-4603-8963-a097994f8eba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:51.126091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060642e-674e-4603-8963-a097994f8eba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:51.127556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060642e-674e-4603-8963-a097994f8eba None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:51.141846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-9060642e-674e-4603-8963-a097994f8eba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:02:51.141846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:02:51.141846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:02:51.141846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:02:51.141846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:02:51.141846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:02:51.141846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:02:51.141846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:02:51.141846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:02:51.141846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:02:51.141846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:02:51.141846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:02:51.141846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:02:51.141846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:02:51.141846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:02:51.143127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:02:51.144047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:02:51.144047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:02:51.144047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:02:51.144047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:02:51.144047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:02:51.144047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:02:51.144047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:02:51.144047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:02:51.148247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 608/1217] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:02:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:02:51.149141 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f9430-b3f7-4a5d-8280-e4bc0a131cbe None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:51.269940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f9430-b3f7-4a5d-8280-e4bc0a131cbe None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:51.294310 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:51.310495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97b0d9-eed4-4f85-9415-5208ae1169ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:51.311195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97b0d9-eed4-4f85-9415-5208ae1169ba None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:51.311487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97b0d9-eed4-4f85-9415-5208ae1169ba None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:51.311789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97b0d9-eed4-4f85-9415-5208ae1169ba None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:51.312720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97b0d9-eed4-4f85-9415-5208ae1169ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:02:51.313067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97b0d9-eed4-4f85-9415-5208ae1169ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:51.314908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97b0d9-eed4-4f85-9415-5208ae1169ba None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:51.332467 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 610/1218] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:02:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:02:51.339605 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060642e-674e-4603-8963-a097994f8eba None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:51.466937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060642e-674e-4603-8963-a097994f8eba None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:51.498591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:51.505812 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621bf066-c60a-4d6e-888a-ee72390167d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:51.506481 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bf066-c60a-4d6e-888a-ee72390167d2 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:51.506806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bf066-c60a-4d6e-888a-ee72390167d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:51.507093 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bf066-c60a-4d6e-888a-ee72390167d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:51.507931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621bf066-c60a-4d6e-888a-ee72390167d2 None admin] RBAC: Adding query filter params (role.id=ae949d8bea4b40d9b1a9fa207d539663, scope.project.id=a498284274454d25b1b118a88996e522, user.id=066f69a87f5f48c7b20c1e0fcf5c1107) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:02:51.508247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621bf066-c60a-4d6e-888a-ee72390167d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:51.510219 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621bf066-c60a-4d6e-888a-ee72390167d2 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:51.523740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 609/1219] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:02:51 2022] GET /identity/v3/role_assignments?user.id=066f69a87f5f48c7b20c1e0fcf5c1107&scope.project.id=a498284274454d25b1b118a88996e522&role.id=ae949d8bea4b40d9b1a9fa207d539663 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:02:53.570259 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883460ef-aa66-417a-bd78-6f2169fa17eb None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:53.570834 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883460ef-aa66-417a-bd78-6f2169fa17eb None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:53.570834 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883460ef-aa66-417a-bd78-6f2169fa17eb None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:53.573078 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 611/1220] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:02:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:02:53.578150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174a08e-7618-4170-8208-27a7d964d3b1 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:53.578566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174a08e-7618-4170-8208-27a7d964d3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:53.578754 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174a08e-7618-4170-8208-27a7d964d3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:53.596049 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-f174a08e-7618-4170-8208-27a7d964d3b1 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:02:53.597230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f174a08e-7618-4170-8208-27a7d964d3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:02:53.668876 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 610/1221] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:02:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 09:02:53.690952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883460ef-aa66-417a-bd78-6f2169fa17eb None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:53.797097 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883460ef-aa66-417a-bd78-6f2169fa17eb None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:53.831981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a5cce-4b32-4334-8b52-164d50c9abb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:53.832697 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a5cce-4b32-4334-8b52-164d50c9abb7 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:53.832973 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a5cce-4b32-4334-8b52-164d50c9abb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:53.833243 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a5cce-4b32-4334-8b52-164d50c9abb7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:53.844170 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a5cce-4b32-4334-8b52-164d50c9abb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:02:53.852752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-4a6a5cce-4b32-4334-8b52-164d50c9abb7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:02:53.852752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:02:53.852752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:02:53.852752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:02:53.852752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:02:53.852752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:02:53.852752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:02:53.852752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:02:53.852752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:02:53.852752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:02:53.852752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:02:53.852752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:02:53.852752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:02:53.852752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:02:53.852752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:02:53.852752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:02:53.853828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:02:53.853828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:02:53.853828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:02:53.853828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:02:53.853828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:02:53.853828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:02:53.853828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:02:53.853828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:02:53.853828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:02:53.853828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:02:53.853828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:02:53.853828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:02:53.853828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:02:53.853828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:02:53.853828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:02:53.853828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:02:53.855140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:02:53.855140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:02:53.855140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:02:53.855140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:02:53.855140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:02:53.855140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:02:53.855140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:02:53.855140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:02:53.855140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:02:53.855140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:02:53.855140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:02:53.855140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:02:53.855140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 612/1222] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:02:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:02:53.860552 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174a08e-7618-4170-8208-27a7d964d3b1 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:53.971402 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174a08e-7618-4170-8208-27a7d964d3b1 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:53.999226 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:54.011755 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f48c3-9171-4e1f-8b81-d3b57b5beca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:54.012570 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f48c3-9171-4e1f-8b81-d3b57b5beca9 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:54.012860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f48c3-9171-4e1f-8b81-d3b57b5beca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:54.013218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f48c3-9171-4e1f-8b81-d3b57b5beca9 None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:54.014020 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f48c3-9171-4e1f-8b81-d3b57b5beca9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:02:54.014371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f48c3-9171-4e1f-8b81-d3b57b5beca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:54.016012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f48c3-9171-4e1f-8b81-d3b57b5beca9 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:54.026039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 611/1223] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 09:02:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:02:54.032069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a5cce-4b32-4334-8b52-164d50c9abb7 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:54.081091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:54.139161 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a5cce-4b32-4334-8b52-164d50c9abb7 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:54.176559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0ac7e-0568-48ae-aeee-d39502abb570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:54.177579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0ac7e-0568-48ae-aeee-d39502abb570 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:54.177906 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0ac7e-0568-48ae-aeee-d39502abb570 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:54.178202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0ac7e-0568-48ae-aeee-d39502abb570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:54.189936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0ac7e-0568-48ae-aeee-d39502abb570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:54.191639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d0ac7e-0568-48ae-aeee-d39502abb570 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:54.213228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-d8d0ac7e-0568-48ae-aeee-d39502abb570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:02:54.213228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:02:54.213228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:02:54.213228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:02:54.213228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:02:54.213228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:02:54.213228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:02:54.213228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:02:54.213228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:02:54.213228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:02:54.213228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:02:54.213228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:02:54.213228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:02:54.213228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:02:54.213228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:02:54.214400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:02:54.215386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:02:54.215386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:02:54.215386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:02:54.215386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:02:54.215386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 613/1224] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:02:54.221186 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f48c3-9171-4e1f-8b81-d3b57b5beca9 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:54.343296 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f48c3-9171-4e1f-8b81-d3b57b5beca9 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:54.388002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6cede-79dd-421d-aead-d8fc057a4afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:54.389090 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6cede-79dd-421d-aead-d8fc057a4afe None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:54.389616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6cede-79dd-421d-aead-d8fc057a4afe None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:54.390063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6cede-79dd-421d-aead-d8fc057a4afe None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:54.391103 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6cede-79dd-421d-aead-d8fc057a4afe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:02:54.391504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6cede-79dd-421d-aead-d8fc057a4afe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:54.393059 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6cede-79dd-421d-aead-d8fc057a4afe None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:54.405493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 612/1225] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:02:54.412071 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0ac7e-0568-48ae-aeee-d39502abb570 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:54.514827 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:54.522306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0ac7e-0568-48ae-aeee-d39502abb570 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:54.558846 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60cf2a-f479-4645-9926-985719db3777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:54.559660 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60cf2a-f479-4645-9926-985719db3777 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:54.559944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60cf2a-f479-4645-9926-985719db3777 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:54.560219 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60cf2a-f479-4645-9926-985719db3777 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:54.568135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60cf2a-f479-4645-9926-985719db3777 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:54.569644 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60cf2a-f479-4645-9926-985719db3777 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:54.584423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-0b60cf2a-f479-4645-9926-985719db3777 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:02:54.584423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:02:54.584423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:02:54.584423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:02:54.584423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:02:54.584423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:02:54.584423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:02:54.584423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:02:54.584423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:02:54.584423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:02:54.584423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:02:54.584423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:02:54.584423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:02:54.584423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:02:54.584423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:02:54.584423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:02:54.585985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:02:54.586986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:02:54.586986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:02:54.586986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:02:54.586986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:02:54.586986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:02:54.586986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:02:54.586986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:02:54.586986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:02:54.586986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:02:54.586986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:02:54.586986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 614/1226] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:02:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:02:54.591908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6cede-79dd-421d-aead-d8fc057a4afe None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:54.643013 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:54.700822 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6cede-79dd-421d-aead-d8fc057a4afe None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:54.740759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312db1f2-a394-4513-9d3d-087d57ae5ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:54.741488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312db1f2-a394-4513-9d3d-087d57ae5ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:54.741808 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312db1f2-a394-4513-9d3d-087d57ae5ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:54.742128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312db1f2-a394-4513-9d3d-087d57ae5ff9 None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:54.743253 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312db1f2-a394-4513-9d3d-087d57ae5ff9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:02:54.743746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312db1f2-a394-4513-9d3d-087d57ae5ff9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:54.745577 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312db1f2-a394-4513-9d3d-087d57ae5ff9 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:54.765608 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 613/1227] 10.222.0.44 () {56 vars in 1143 bytes} [Wed Jul 20 09:02:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:02:54.772266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60cf2a-f479-4645-9926-985719db3777 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:54.877777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60cf2a-f479-4645-9926-985719db3777 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:54.913039 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67e61f-8078-403e-a60e-0d5a1534a883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:54.913874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67e61f-8078-403e-a60e-0d5a1534a883 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:54.914170 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67e61f-8078-403e-a60e-0d5a1534a883 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:54.914610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67e61f-8078-403e-a60e-0d5a1534a883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:54.928085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67e61f-8078-403e-a60e-0d5a1534a883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:54.929727 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67e61f-8078-403e-a60e-0d5a1534a883 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:54.950470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-cb67e61f-8078-403e-a60e-0d5a1534a883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:02:54.950470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:02:54.950470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:02:54.950470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:02:54.950470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:02:54.950470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:02:54.950470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:02:54.950470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:02:54.950470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:02:54.950470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:02:54.950470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:02:54.950470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:02:54.950470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:02:54.950470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:02:54.950470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:02:54.951626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:02:54.952620 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:02:54.952620 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:02:54.952620 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:02:54.952620 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:02:54.952620 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 615/1228] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:02:54.957846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312db1f2-a394-4513-9d3d-087d57ae5ff9 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:55.077128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312db1f2-a394-4513-9d3d-087d57ae5ff9 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:55.079599 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:55.114560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320b2d8-9826-4f9b-b16a-8ccb3b28a0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:55.115379 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320b2d8-9826-4f9b-b16a-8ccb3b28a0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:55.115634 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320b2d8-9826-4f9b-b16a-8ccb3b28a0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:55.115913 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320b2d8-9826-4f9b-b16a-8ccb3b28a0e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:55.116687 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320b2d8-9826-4f9b-b16a-8ccb3b28a0e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:02:55.116956 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320b2d8-9826-4f9b-b16a-8ccb3b28a0e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:55.118312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320b2d8-9826-4f9b-b16a-8ccb3b28a0e1 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:55.136216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 614/1229] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:02:55.137582 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67e61f-8078-403e-a60e-0d5a1534a883 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:55.148020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:55.248017 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67e61f-8078-403e-a60e-0d5a1534a883 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:55.285530 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480cbf3-72f8-4b2b-9e3f-69bb81b67f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:55.286495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480cbf3-72f8-4b2b-9e3f-69bb81b67f42 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:55.286893 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480cbf3-72f8-4b2b-9e3f-69bb81b67f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:55.287422 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480cbf3-72f8-4b2b-9e3f-69bb81b67f42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:55.296322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d480cbf3-72f8-4b2b-9e3f-69bb81b67f42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:55.298137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d480cbf3-72f8-4b2b-9e3f-69bb81b67f42 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:55.315540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-d480cbf3-72f8-4b2b-9e3f-69bb81b67f42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:02:55.315540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:02:55.315540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:02:55.315540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:02:55.315540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:02:55.315540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:02:55.315540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:02:55.315540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:02:55.315540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:02:55.315540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:02:55.315540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:02:55.315540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:02:55.315540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:02:55.315540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:02:55.315540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:02:55.316613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:02:55.317619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:02:55.317619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:02:55.317619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:02:55.317619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:02:55.317619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:02:55.317619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:02:55.317619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:02:55.317619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:02:55.317619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 616/1230] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:02:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:02:55.324976 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320b2d8-9826-4f9b-b16a-8ccb3b28a0e1 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:55.440178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320b2d8-9826-4f9b-b16a-8ccb3b28a0e1 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:55.479887 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca9384-031f-4419-9a9c-fa2d63d6d775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:55.480766 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca9384-031f-4419-9a9c-fa2d63d6d775 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:55.481030 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca9384-031f-4419-9a9c-fa2d63d6d775 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:55.481406 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca9384-031f-4419-9a9c-fa2d63d6d775 None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:55.482207 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca9384-031f-4419-9a9c-fa2d63d6d775 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:02:55.482563 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca9384-031f-4419-9a9c-fa2d63d6d775 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:55.484158 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca9384-031f-4419-9a9c-fa2d63d6d775 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:55.498132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 615/1231] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:02:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:02:55.504906 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480cbf3-72f8-4b2b-9e3f-69bb81b67f42 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:55.600624 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:55.641866 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480cbf3-72f8-4b2b-9e3f-69bb81b67f42 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:55.679692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3f961-7dac-4ace-a642-004a350c266b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:55.680717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3f961-7dac-4ace-a642-004a350c266b None admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:55.681911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3f961-7dac-4ace-a642-004a350c266b None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:55.682353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3f961-7dac-4ace-a642-004a350c266b None admin] PATH_INFO: `/v3/projects/a498284274454d25b1b118a88996e522/users/066f69a87f5f48c7b20c1e0fcf5c1107/roles/ae949d8bea4b40d9b1a9fa207d539663` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:55.699082 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3f961-7dac-4ace-a642-004a350c266b None admin] RBAC: Authorizing `identity:create_grant(project_id=a498284274454d25b1b118a88996e522, user_id=066f69a87f5f48c7b20c1e0fcf5c1107, role_id=ae949d8bea4b40d9b1a9fa207d539663)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:55.700669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3f961-7dac-4ace-a642-004a350c266b None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:55.715863 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 617/1232] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 09:02:55 2022] PUT /identity/v3/projects/a498284274454d25b1b118a88996e522/users/066f69a87f5f48c7b20c1e0fcf5c1107/roles/ae949d8bea4b40d9b1a9fa207d539663 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:02:57.917852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe450df-e58f-4cbe-a5e6-678c11216e77 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:57.918481 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe450df-e58f-4cbe-a5e6-678c11216e77 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:57.918481 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe450df-e58f-4cbe-a5e6-678c11216e77 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:57.921171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 616/1233] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:02:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:02:57.928584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc933910-663a-4bb6-afe6-2181b3a02272 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:57.928992 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc933910-663a-4bb6-afe6-2181b3a02272 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:57.929163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc933910-663a-4bb6-afe6-2181b3a02272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:57.946249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-bc933910-663a-4bb6-afe6-2181b3a02272 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:02:57.947702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-bc933910-663a-4bb6-afe6-2181b3a02272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:02:58.018474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 618/1234] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:02:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:02:58.043487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe450df-e58f-4cbe-a5e6-678c11216e77 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:58.069259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:58.156062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe450df-e58f-4cbe-a5e6-678c11216e77 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:58.192430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7ed19-caf5-4a8e-8547-cb96139786a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:58.193132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7ed19-caf5-4a8e-8547-cb96139786a4 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:58.193449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7ed19-caf5-4a8e-8547-cb96139786a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:58.193693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7ed19-caf5-4a8e-8547-cb96139786a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:58.205359 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb7ed19-caf5-4a8e-8547-cb96139786a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:02:58.214066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-ffb7ed19-caf5-4a8e-8547-cb96139786a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:02:58.214066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:02:58.214066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:02:58.214066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:02:58.214066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:02:58.214066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:02:58.214066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:02:58.214066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:02:58.214066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:02:58.214066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:02:58.214066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:02:58.214066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:02:58.214066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:02:58.214066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:02:58.214066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:02:58.214066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:02:58.215232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:02:58.215232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:02:58.215232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:02:58.215232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:02:58.215232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:02:58.215232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:02:58.215232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:02:58.215232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:02:58.215232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:02:58.215232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:02:58.215232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:02:58.215232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:02:58.215232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:02:58.215232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:02:58.215232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:02:58.215232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:02:58.216107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:02:58.216107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:02:58.216107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:02:58.216107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:02:58.216107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:02:58.216107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:02:58.216107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:02:58.216107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:02:58.216107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:02:58.216107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:02:58.216107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 09:02:58.216107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:02:58.216107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 617/1235] 10.222.0.44 () {56 vars in 1102 bytes} [Wed Jul 20 09:02:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:02:58.221692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc933910-663a-4bb6-afe6-2181b3a02272 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:58.273812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:58.337932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc933910-663a-4bb6-afe6-2181b3a02272 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:58.373748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952711f4-ed00-4a6b-aaba-b4a3510a0ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:58.374418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952711f4-ed00-4a6b-aaba-b4a3510a0ffe None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:58.374757 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952711f4-ed00-4a6b-aaba-b4a3510a0ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:58.375020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952711f4-ed00-4a6b-aaba-b4a3510a0ffe None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:58.375803 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952711f4-ed00-4a6b-aaba-b4a3510a0ffe None admin] RBAC: Adding query filter params (name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:02:58.376090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952711f4-ed00-4a6b-aaba-b4a3510a0ffe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:58.377453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952711f4-ed00-4a6b-aaba-b4a3510a0ffe None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:58.387192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 619/1236] 10.222.0.44 () {56 vars in 1103 bytes} [Wed Jul 20 09:02:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:02:58.392959 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7ed19-caf5-4a8e-8547-cb96139786a4 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:58.486208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:58.504968 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7ed19-caf5-4a8e-8547-cb96139786a4 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:58.542481 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff823b-2fbc-442a-b76a-358b1fdc90df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:58.543315 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff823b-2fbc-442a-b76a-358b1fdc90df None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:58.543623 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff823b-2fbc-442a-b76a-358b1fdc90df None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:58.543935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff823b-2fbc-442a-b76a-358b1fdc90df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:58.557720 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff823b-2fbc-442a-b76a-358b1fdc90df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:58.559216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff823b-2fbc-442a-b76a-358b1fdc90df None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:58.578473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-60ff823b-2fbc-442a-b76a-358b1fdc90df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:02:58.578473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:02:58.578473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:02:58.578473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:02:58.578473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:02:58.578473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:02:58.578473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:02:58.578473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:02:58.578473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:02:58.578473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:02:58.578473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:02:58.578473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:02:58.578473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:02:58.578473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:02:58.578473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:02:58.579611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:02:58.580760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:02:58.580760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:02:58.580760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:02:58.580760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:02:58.580760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 618/1237] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:02:58.586009 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952711f4-ed00-4a6b-aaba-b4a3510a0ffe None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:58.693655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952711f4-ed00-4a6b-aaba-b4a3510a0ffe None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:58.703697 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:58.729568 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5554d4-3ad9-4e3e-b432-1b6d327fd4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:58.730239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5554d4-3ad9-4e3e-b432-1b6d327fd4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:58.730526 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5554d4-3ad9-4e3e-b432-1b6d327fd4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:58.730907 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5554d4-3ad9-4e3e-b432-1b6d327fd4c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:58.731636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5554d4-3ad9-4e3e-b432-1b6d327fd4c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:02:58.731944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5554d4-3ad9-4e3e-b432-1b6d327fd4c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:58.733309 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5554d4-3ad9-4e3e-b432-1b6d327fd4c3 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:58.745396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 620/1238] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:02:58.751122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff823b-2fbc-442a-b76a-358b1fdc90df None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:58.861191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff823b-2fbc-442a-b76a-358b1fdc90df None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:58.905083 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27930a1-b378-47cd-a1d7-b47f4d1a3b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:58.906078 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27930a1-b378-47cd-a1d7-b47f4d1a3b35 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:58.906367 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27930a1-b378-47cd-a1d7-b47f4d1a3b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:58.906654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27930a1-b378-47cd-a1d7-b47f4d1a3b35 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:58.909810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:58.916778 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27930a1-b378-47cd-a1d7-b47f4d1a3b35 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:58.919048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27930a1-b378-47cd-a1d7-b47f4d1a3b35 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:58.935099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-f27930a1-b378-47cd-a1d7-b47f4d1a3b35 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:02:58.935099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:02:58.935099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:02:58.935099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:02:58.935099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:02:58.935099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:02:58.935099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:02:58.935099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:02:58.935099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:02:58.935099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:02:58.935099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:02:58.935099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:02:58.935099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:02:58.935099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:02:58.935099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:02:58.935099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:02:58.936328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:02:58.937282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:02:58.937282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:02:58.937282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:02:58.937282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:02:58.937282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:02:58.937282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:02:58.937282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:02:58.937282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:02:58.937282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:02:58.937282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:02:58.937282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 619/1239] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:02:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:02:58.942308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5554d4-3ad9-4e3e-b432-1b6d327fd4c3 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:59.047735 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5554d4-3ad9-4e3e-b432-1b6d327fd4c3 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:59.084240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185cbfc-8fc0-4365-957f-0ad121963753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:59.085108 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185cbfc-8fc0-4365-957f-0ad121963753 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:59.085324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185cbfc-8fc0-4365-957f-0ad121963753 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:59.085672 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185cbfc-8fc0-4365-957f-0ad121963753 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:59.086517 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185cbfc-8fc0-4365-957f-0ad121963753 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:02:59.086869 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185cbfc-8fc0-4365-957f-0ad121963753 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:59.088707 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7185cbfc-8fc0-4365-957f-0ad121963753 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:59.108540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 621/1240] 10.222.0.44 () {56 vars in 1143 bytes} [Wed Jul 20 09:02:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:02:59.115041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27930a1-b378-47cd-a1d7-b47f4d1a3b35 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:59.220063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27930a1-b378-47cd-a1d7-b47f4d1a3b35 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:59.260314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522f0ef-9c76-4e27-a250-4c8f8e8a5b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:59.260970 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522f0ef-9c76-4e27-a250-4c8f8e8a5b7d None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:59.261294 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522f0ef-9c76-4e27-a250-4c8f8e8a5b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:59.261740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522f0ef-9c76-4e27-a250-4c8f8e8a5b7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:59.348217 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:59.357328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522f0ef-9c76-4e27-a250-4c8f8e8a5b7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:59.359361 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522f0ef-9c76-4e27-a250-4c8f8e8a5b7d None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:59.470913 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 620/1241] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:02:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 09:02:59.476652 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185cbfc-8fc0-4365-957f-0ad121963753 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:59.511742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:59.593584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7185cbfc-8fc0-4365-957f-0ad121963753 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:59.628604 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6235d-dd98-46f9-804c-82fbbf5725c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:59.629311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6235d-dd98-46f9-804c-82fbbf5725c3 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:59.629598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6235d-dd98-46f9-804c-82fbbf5725c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:59.629916 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6235d-dd98-46f9-804c-82fbbf5725c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:59.641121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6235d-dd98-46f9-804c-82fbbf5725c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:59.642715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6235d-dd98-46f9-804c-82fbbf5725c3 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:59.661837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-a5e6235d-dd98-46f9-804c-82fbbf5725c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:02:59.661837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:02:59.661837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:02:59.661837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:02:59.661837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:02:59.661837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:02:59.661837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:02:59.661837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:02:59.661837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:02:59.661837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:02:59.661837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:02:59.661837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:02:59.661837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:02:59.661837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:02:59.661837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:02:59.662831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:02:59.663747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:02:59.663747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:02:59.663747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:02:59.663747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:02:59.663747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 622/1242] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:02:59.669225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522f0ef-9c76-4e27-a250-4c8f8e8a5b7d None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:59.772367 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:59.779567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522f0ef-9c76-4e27-a250-4c8f8e8a5b7d None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:59.815945 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d481fe1-1d81-442e-a57d-9ce4226a77a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:59.816614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d481fe1-1d81-442e-a57d-9ce4226a77a2 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:59.816901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d481fe1-1d81-442e-a57d-9ce4226a77a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:59.817178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d481fe1-1d81-442e-a57d-9ce4226a77a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:02:59.817904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d481fe1-1d81-442e-a57d-9ce4226a77a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:02:59.818215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d481fe1-1d81-442e-a57d-9ce4226a77a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:02:59.819619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d481fe1-1d81-442e-a57d-9ce4226a77a2 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:02:59.831141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 621/1243] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:02:59.836797 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6235d-dd98-46f9-804c-82fbbf5725c3 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:02:59.924990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:02:59.951249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6235d-dd98-46f9-804c-82fbbf5725c3 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:02:59.989041 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bf9f1-2fc9-42f6-9693-c5984aee68f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:02:59.989899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bf9f1-2fc9-42f6-9693-c5984aee68f2 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:02:59.990180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bf9f1-2fc9-42f6-9693-c5984aee68f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:02:59.990489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bf9f1-2fc9-42f6-9693-c5984aee68f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:00.001048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123bf9f1-2fc9-42f6-9693-c5984aee68f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:00.002605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123bf9f1-2fc9-42f6-9693-c5984aee68f2 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:00.019286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-123bf9f1-2fc9-42f6-9693-c5984aee68f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:03:00.019286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:00.019286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:00.019286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:00.019286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:00.019286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:00.019286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:00.019286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:00.019286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:00.019286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:00.019286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:00.019286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:00.019286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:03:00.019286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:03:00.019286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:00.020242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:00.021137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:03:00.021137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:03:00.021137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:03:00.021137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:03:00.021137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:03:00.021137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:03:00.021137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:03:00.021137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:03:00.021137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 623/1244] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:02:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:00.026927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d481fe1-1d81-442e-a57d-9ce4226a77a2 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:00.140382 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d481fe1-1d81-442e-a57d-9ce4226a77a2 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:00.176931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f0fbb-86b0-44ef-bb9c-adc1aaa7de27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:00.177519 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f0fbb-86b0-44ef-bb9c-adc1aaa7de27 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:00.177841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f0fbb-86b0-44ef-bb9c-adc1aaa7de27 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:00.178162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f0fbb-86b0-44ef-bb9c-adc1aaa7de27 None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:00.179046 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f0fbb-86b0-44ef-bb9c-adc1aaa7de27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:00.179377 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f0fbb-86b0-44ef-bb9c-adc1aaa7de27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:00.181393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f0fbb-86b0-44ef-bb9c-adc1aaa7de27 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:00.195776 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 622/1245] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:03:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:00.202551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bf9f1-2fc9-42f6-9693-c5984aee68f2 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:00.313748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bf9f1-2fc9-42f6-9693-c5984aee68f2 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:00.357489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3a42c-fb50-4d8b-bbfc-dcdcd40bf8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:00.358314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3a42c-fb50-4d8b-bbfc-dcdcd40bf8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:00.358607 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3a42c-fb50-4d8b-bbfc-dcdcd40bf8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:00.359039 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3a42c-fb50-4d8b-bbfc-dcdcd40bf8a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:00.359910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3a42c-fb50-4d8b-bbfc-dcdcd40bf8a6 None admin] RBAC: Adding query filter params (role.id=ae949d8bea4b40d9b1a9fa207d539663, scope.project.id=a498284274454d25b1b118a88996e522, user.id=066f69a87f5f48c7b20c1e0fcf5c1107) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:00.360228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3a42c-fb50-4d8b-bbfc-dcdcd40bf8a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:00.361811 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3a42c-fb50-4d8b-bbfc-dcdcd40bf8a6 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:00.367737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 624/1246] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:03:00 2022] GET /identity/v3/role_assignments?user.id=066f69a87f5f48c7b20c1e0fcf5c1107&scope.project.id=a498284274454d25b1b118a88996e522&role.id=ae949d8bea4b40d9b1a9fa207d539663 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:02.503887 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933029d-bce6-46c3-9200-db0101efa8ae None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:02.504427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933029d-bce6-46c3-9200-db0101efa8ae None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:02.504427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933029d-bce6-46c3-9200-db0101efa8ae None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:02.505821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 623/1247] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:03:02.511683 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0b5d1-ce69-492e-9ebf-335e087d3594 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:02.512106 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0b5d1-ce69-492e-9ebf-335e087d3594 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:02.512247 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d0b5d1-ce69-492e-9ebf-335e087d3594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:02.517282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:02.530138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-10d0b5d1-ce69-492e-9ebf-335e087d3594 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:03:02.531370 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-10d0b5d1-ce69-492e-9ebf-335e087d3594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:03:02.606749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 625/1248] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:03:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:03:02.628744 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933029d-bce6-46c3-9200-db0101efa8ae None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:02.647856 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:02.742760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933029d-bce6-46c3-9200-db0101efa8ae None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:02.783837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1eedbf-07dc-41c1-8214-c349e1e197fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:02.784555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1eedbf-07dc-41c1-8214-c349e1e197fb None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:02.784834 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1eedbf-07dc-41c1-8214-c349e1e197fb None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:02.785108 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1eedbf-07dc-41c1-8214-c349e1e197fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:02.797380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1eedbf-07dc-41c1-8214-c349e1e197fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:03:02.806612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-0a1eedbf-07dc-41c1-8214-c349e1e197fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:03:02.806612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:02.806612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:02.806612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:02.806612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:02.806612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:02.806612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:02.806612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:02.806612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:02.806612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:02.806612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:02.806612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:02.806612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:03:02.806612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:03:02.806612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:03:02.806612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:03:02.807636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:03:02.807636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:03:02.807636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:03:02.807636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:03:02.807636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:02.807636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:02.807636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:02.807636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:02.807636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:02.807636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:02.807636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:02.807636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:02.807636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:02.807636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:02.807636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:02.807636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:02.808516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:02.808516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:02.808516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:02.808516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:02.808516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:03:02.808516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:03:02.808516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:03:02.808516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:03:02.808516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:03:02.808516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:03:02.808516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:03:02.808516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:03:02.808516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 624/1249] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:03:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:02.814898 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0b5d1-ce69-492e-9ebf-335e087d3594 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:02.927916 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d0b5d1-ce69-492e-9ebf-335e087d3594 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:02.960476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:02.966767 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118046a-6ce8-41d2-a536-33b57f6e04b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:02.967464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118046a-6ce8-41d2-a536-33b57f6e04b6 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:02.968125 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118046a-6ce8-41d2-a536-33b57f6e04b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:02.968420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118046a-6ce8-41d2-a536-33b57f6e04b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:02.969221 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118046a-6ce8-41d2-a536-33b57f6e04b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:02.969536 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118046a-6ce8-41d2-a536-33b57f6e04b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:02.970939 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9118046a-6ce8-41d2-a536-33b57f6e04b6 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:02.981672 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 626/1250] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:03:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:02.987837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1eedbf-07dc-41c1-8214-c349e1e197fb None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:03.071074 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:03.098204 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1eedbf-07dc-41c1-8214-c349e1e197fb None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:03.136262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f5f0b-1047-4e00-ba28-ea1fe0d62f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:03.137182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f5f0b-1047-4e00-ba28-ea1fe0d62f24 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:03.137504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f5f0b-1047-4e00-ba28-ea1fe0d62f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:03.137801 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f5f0b-1047-4e00-ba28-ea1fe0d62f24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:03.149903 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f5f0b-1047-4e00-ba28-ea1fe0d62f24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:03.151568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f5f0b-1047-4e00-ba28-ea1fe0d62f24 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:03.170150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-574f5f0b-1047-4e00-ba28-ea1fe0d62f24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:03:03.170150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:03.170150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:03.170150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:03.170150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:03.170150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:03.170150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:03.170150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:03.170150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:03.170150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:03.170150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:03.170150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:03.170150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:03:03.170150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:03:03.170150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:03.171164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:03.172068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:03:03.172068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:03:03.172068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:03:03.172068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:03:03.172068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 625/1251] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:03.177369 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118046a-6ce8-41d2-a536-33b57f6e04b6 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:03.293097 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118046a-6ce8-41d2-a536-33b57f6e04b6 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:03.336918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8a4c8-40f7-4e7f-a9fe-ab2240781248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:03.338200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8a4c8-40f7-4e7f-a9fe-ab2240781248 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:03.338494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8a4c8-40f7-4e7f-a9fe-ab2240781248 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:03.338854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec8a4c8-40f7-4e7f-a9fe-ab2240781248 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:03.339672 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8a4c8-40f7-4e7f-a9fe-ab2240781248 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:03.339980 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8a4c8-40f7-4e7f-a9fe-ab2240781248 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:03.341684 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec8a4c8-40f7-4e7f-a9fe-ab2240781248 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:03.360334 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 627/1252] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:03:03.361573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f5f0b-1047-4e00-ba28-ea1fe0d62f24 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:03.472856 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f5f0b-1047-4e00-ba28-ea1fe0d62f24 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:03.504983 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:03.514057 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b530d-d8d8-4850-98af-dcdb23cfb376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:03.514893 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b530d-d8d8-4850-98af-dcdb23cfb376 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:03.515177 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b530d-d8d8-4850-98af-dcdb23cfb376 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:03.515454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b530d-d8d8-4850-98af-dcdb23cfb376 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:03.523075 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b530d-d8d8-4850-98af-dcdb23cfb376 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:03.525100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b530d-d8d8-4850-98af-dcdb23cfb376 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:03.539665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-928b530d-d8d8-4850-98af-dcdb23cfb376 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:03:03.539665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:03.539665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:03.539665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:03.539665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:03.539665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:03.539665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:03.539665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:03.539665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:03.539665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:03.539665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:03.539665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:03.539665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:03:03.539665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:03:03.539665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:03:03.539665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:03.540852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:03.542034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:03.542034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:03.542034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:03:03.542034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:03:03.542034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:03:03.542034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:03:03.542034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:03:03.542034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:03:03.542034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:03:03.542034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:03:03.542034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 626/1253] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:03:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:03.547002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8a4c8-40f7-4e7f-a9fe-ab2240781248 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:03.611940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:03.660877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec8a4c8-40f7-4e7f-a9fe-ab2240781248 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:03.696305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1318269-7f15-4abe-a714-dfb14b307bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:03.697544 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1318269-7f15-4abe-a714-dfb14b307bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:03.697814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1318269-7f15-4abe-a714-dfb14b307bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:03.698086 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1318269-7f15-4abe-a714-dfb14b307bd6 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:03.698956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1318269-7f15-4abe-a714-dfb14b307bd6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:03.699259 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1318269-7f15-4abe-a714-dfb14b307bd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:03.700687 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1318269-7f15-4abe-a714-dfb14b307bd6 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:03.721498 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 628/1254] 10.222.0.44 () {56 vars in 1143 bytes} [Wed Jul 20 09:03:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:03.727419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b530d-d8d8-4850-98af-dcdb23cfb376 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:03.834708 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b530d-d8d8-4850-98af-dcdb23cfb376 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:03.870941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6100ae-820a-4570-bf4a-cb4d08b8efdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:03.871519 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6100ae-820a-4570-bf4a-cb4d08b8efdc None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:03.871805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6100ae-820a-4570-bf4a-cb4d08b8efdc None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:03.872073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6100ae-820a-4570-bf4a-cb4d08b8efdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:03.929508 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:03.943892 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6100ae-820a-4570-bf4a-cb4d08b8efdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:03.945735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6100ae-820a-4570-bf4a-cb4d08b8efdc None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:04.059366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 627/1255] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:03:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:03:04.061283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1318269-7f15-4abe-a714-dfb14b307bd6 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:04.180146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1318269-7f15-4abe-a714-dfb14b307bd6 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:04.193704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:04.216619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d60a6-9759-4ab3-a0ed-ead9594bec0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:04.217638 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d60a6-9759-4ab3-a0ed-ead9594bec0d None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:04.218038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d60a6-9759-4ab3-a0ed-ead9594bec0d None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:04.218403 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d60a6-9759-4ab3-a0ed-ead9594bec0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:04.229171 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d60a6-9759-4ab3-a0ed-ead9594bec0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:04.230869 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d60a6-9759-4ab3-a0ed-ead9594bec0d None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:04.250513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-cc1d60a6-9759-4ab3-a0ed-ead9594bec0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:03:04.250513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:04.250513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:04.250513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:04.250513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:04.250513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:04.250513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:04.250513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:04.250513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:04.250513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:04.250513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:04.250513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:04.250513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:03:04.250513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:03:04.250513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:04.251572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:04.252493 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:03:04.252493 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:03:04.252493 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:03:04.252493 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:03:04.252493 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 629/1256] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:04.258222 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6100ae-820a-4570-bf4a-cb4d08b8efdc None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:04.364640 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:04.380061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6100ae-820a-4570-bf4a-cb4d08b8efdc None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:04.420910 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9b8d8-5dfe-47c6-ad38-f277ae165718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:04.421557 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9b8d8-5dfe-47c6-ad38-f277ae165718 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:04.421775 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9b8d8-5dfe-47c6-ad38-f277ae165718 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:04.422052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c9b8d8-5dfe-47c6-ad38-f277ae165718 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:04.422792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9b8d8-5dfe-47c6-ad38-f277ae165718 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:04.423096 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9b8d8-5dfe-47c6-ad38-f277ae165718 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:04.424455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c9b8d8-5dfe-47c6-ad38-f277ae165718 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:04.436786 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 628/1257] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:04.444277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d60a6-9759-4ab3-a0ed-ead9594bec0d None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:04.558902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d60a6-9759-4ab3-a0ed-ead9594bec0d None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:04.597919 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38515cc9-176f-4cd8-8dd3-a5aa03a0a334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:04.598762 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38515cc9-176f-4cd8-8dd3-a5aa03a0a334 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:04.599054 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38515cc9-176f-4cd8-8dd3-a5aa03a0a334 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:04.599337 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38515cc9-176f-4cd8-8dd3-a5aa03a0a334 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:04.606559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38515cc9-176f-4cd8-8dd3-a5aa03a0a334 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:04.608047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38515cc9-176f-4cd8-8dd3-a5aa03a0a334 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:04.623143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-38515cc9-176f-4cd8-8dd3-a5aa03a0a334 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:03:04.623143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:04.623143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:04.623143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:04.623143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:04.623143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:04.623143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:04.623143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:04.623143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:04.623143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:04.623143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:04.623143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:04.623143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:03:04.623143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:03:04.623143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:04.624287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:04.625316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:03:04.625316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:03:04.625316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:03:04.625316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:03:04.625316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:03:04.625316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:03:04.625316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:03:04.625316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:03:04.625316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 630/1258] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:03:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:04.630315 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9b8d8-5dfe-47c6-ad38-f277ae165718 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:04.738892 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c9b8d8-5dfe-47c6-ad38-f277ae165718 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:04.776832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2456e22-2939-49ba-8526-c9a9399e4b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:04.777520 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2456e22-2939-49ba-8526-c9a9399e4b8c None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:04.777810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2456e22-2939-49ba-8526-c9a9399e4b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:04.778093 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2456e22-2939-49ba-8526-c9a9399e4b8c None admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:04.778939 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2456e22-2939-49ba-8526-c9a9399e4b8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:04.779255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2456e22-2939-49ba-8526-c9a9399e4b8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:04.780724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2456e22-2939-49ba-8526-c9a9399e4b8c None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:04.794383 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 629/1259] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:03:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:04.800613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38515cc9-176f-4cd8-8dd3-a5aa03a0a334 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:04.828567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:04.907297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38515cc9-176f-4cd8-8dd3-a5aa03a0a334 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:04.943035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11708307-839b-4983-9cdb-4276f2c5282d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:04.943691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11708307-839b-4983-9cdb-4276f2c5282d None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:04.943972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11708307-839b-4983-9cdb-4276f2c5282d None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:04.944245 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11708307-839b-4983-9cdb-4276f2c5282d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:04.945337 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11708307-839b-4983-9cdb-4276f2c5282d None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=a498284274454d25b1b118a88996e522, user.id=066f69a87f5f48c7b20c1e0fcf5c1107) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:04.945644 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11708307-839b-4983-9cdb-4276f2c5282d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:04.948232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11708307-839b-4983-9cdb-4276f2c5282d None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:04.954536 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 631/1260] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:03:04 2022] GET /identity/v3/role_assignments?user.id=066f69a87f5f48c7b20c1e0fcf5c1107&scope.project.id=a498284274454d25b1b118a88996e522&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:06.948847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f982d66-bf77-4427-9003-65a2a0222c45 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:06.949405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f982d66-bf77-4427-9003-65a2a0222c45 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:06.949405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f982d66-bf77-4427-9003-65a2a0222c45 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:06.950802 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 630/1261] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:03:06.955824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564d132-c8f3-4424-8138-646cfaa2964d None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:06.956242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564d132-c8f3-4424-8138-646cfaa2964d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:06.956431 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564d132-c8f3-4424-8138-646cfaa2964d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:06.973596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-c564d132-c8f3-4424-8138-646cfaa2964d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:03:06.974817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c564d132-c8f3-4424-8138-646cfaa2964d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:03:07.041104 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 632/1262] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:03:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:03:07.063756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f982d66-bf77-4427-9003-65a2a0222c45 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:07.075270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:07.171894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f982d66-bf77-4427-9003-65a2a0222c45 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:07.205518 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda12eb-f8eb-4456-b573-006bedca0c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:07.206242 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda12eb-f8eb-4456-b573-006bedca0c7e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:07.206529 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda12eb-f8eb-4456-b573-006bedca0c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:07.206840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda12eb-f8eb-4456-b573-006bedca0c7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:07.217266 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda12eb-f8eb-4456-b573-006bedca0c7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:03:07.224880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-1bda12eb-f8eb-4456-b573-006bedca0c7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:03:07.224880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:07.224880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:07.224880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:07.224880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:07.224880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:07.224880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:07.224880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:07.224880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:07.224880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:07.224880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:07.224880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:07.224880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:03:07.224880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:03:07.224880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:03:07.224880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:03:07.225951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:03:07.225951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:03:07.225951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:03:07.225951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:03:07.225951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:07.225951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:07.225951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:07.225951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:07.225951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:07.225951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:07.225951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:07.225951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:07.225951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:07.225951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:07.225951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:07.225951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:07.226998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:07.226998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:07.226998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:07.226998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:07.226998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:03:07.226998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:03:07.226998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:03:07.226998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:03:07.226998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:03:07.226998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:03:07.226998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:03:07.226998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:03:07.226998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 631/1263] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:03:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:07.232240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564d132-c8f3-4424-8138-646cfaa2964d None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:07.249830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:07.333563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564d132-c8f3-4424-8138-646cfaa2964d None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:07.367666 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65a17e-2e03-4a7f-9466-184ad46f93cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:07.368842 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65a17e-2e03-4a7f-9466-184ad46f93cf None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:07.369290 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65a17e-2e03-4a7f-9466-184ad46f93cf None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:07.369574 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65a17e-2e03-4a7f-9466-184ad46f93cf None admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:07.370318 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65a17e-2e03-4a7f-9466-184ad46f93cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:07.370624 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65a17e-2e03-4a7f-9466-184ad46f93cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:07.372013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65a17e-2e03-4a7f-9466-184ad46f93cf None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:07.379840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 633/1264] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:03:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:07.386429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda12eb-f8eb-4456-b573-006bedca0c7e None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:07.454079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:07.487307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda12eb-f8eb-4456-b573-006bedca0c7e None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:07.521240 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b8cc5-83b6-4125-9242-35b8892cf21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:07.522017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b8cc5-83b6-4125-9242-35b8892cf21a None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:07.522302 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b8cc5-83b6-4125-9242-35b8892cf21a None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:07.522582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b8cc5-83b6-4125-9242-35b8892cf21a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:07.533796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b8cc5-83b6-4125-9242-35b8892cf21a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:07.535326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b8cc5-83b6-4125-9242-35b8892cf21a None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:07.552270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-341b8cc5-83b6-4125-9242-35b8892cf21a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:03:07.552270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:07.552270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:07.552270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:07.552270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:07.552270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:07.552270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:07.552270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:07.552270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:07.552270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:07.552270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:07.552270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:07.552270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:03:07.552270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:03:07.552270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:07.553289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:07.554371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:03:07.554371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:03:07.554371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:03:07.554371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:03:07.554371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 632/1265] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:03:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:07.559967 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65a17e-2e03-4a7f-9466-184ad46f93cf None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:07.639361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:07.657854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65a17e-2e03-4a7f-9466-184ad46f93cf None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:07.690149 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60944d14-a34f-4e51-80b9-bfafa2d179c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:07.690834 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60944d14-a34f-4e51-80b9-bfafa2d179c1 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:07.691119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60944d14-a34f-4e51-80b9-bfafa2d179c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:07.691399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60944d14-a34f-4e51-80b9-bfafa2d179c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:07.692109 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60944d14-a34f-4e51-80b9-bfafa2d179c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:07.692413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60944d14-a34f-4e51-80b9-bfafa2d179c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:07.693760 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60944d14-a34f-4e51-80b9-bfafa2d179c1 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:07.706037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 634/1266] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:07.711801 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b8cc5-83b6-4125-9242-35b8892cf21a None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:07.807196 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b8cc5-83b6-4125-9242-35b8892cf21a None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:07.826386 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:07.840497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d860b-7f9a-415c-a998-820cf48081f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:07.841280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d860b-7f9a-415c-a998-820cf48081f7 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:07.841559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d860b-7f9a-415c-a998-820cf48081f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:07.841835 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d860b-7f9a-415c-a998-820cf48081f7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:07.848995 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d860b-7f9a-415c-a998-820cf48081f7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:07.850414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d860b-7f9a-415c-a998-820cf48081f7 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:07.864125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-2b2d860b-7f9a-415c-a998-820cf48081f7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:03:07.864125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:07.864125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:07.864125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:07.864125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:07.864125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:07.864125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:07.864125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:07.864125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:07.864125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:07.864125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:07.864125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:07.864125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:03:07.864125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:03:07.864125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:03:07.864125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:07.865268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:07.866455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:07.866455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:07.866455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:03:07.866455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:03:07.866455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:03:07.866455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:03:07.866455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:03:07.866455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:03:07.866455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:03:07.866455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:03:07.866455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 633/1267] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:03:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:07.873271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60944d14-a34f-4e51-80b9-bfafa2d179c1 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:07.968892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60944d14-a34f-4e51-80b9-bfafa2d179c1 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:08.000682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30979c67-cc4a-4291-acb5-c2cd38b2c9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:08.001327 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30979c67-cc4a-4291-acb5-c2cd38b2c9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:08.001605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30979c67-cc4a-4291-acb5-c2cd38b2c9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:08.001879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30979c67-cc4a-4291-acb5-c2cd38b2c9a1 None admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:08.002722 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30979c67-cc4a-4291-acb5-c2cd38b2c9a1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:08.003032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30979c67-cc4a-4291-acb5-c2cd38b2c9a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:08.004437 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30979c67-cc4a-4291-acb5-c2cd38b2c9a1 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:08.013278 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:08.022021 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 635/1268] 10.222.0.44 () {56 vars in 1143 bytes} [Wed Jul 20 09:03:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:08.028707 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d860b-7f9a-415c-a998-820cf48081f7 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:08.124883 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d860b-7f9a-415c-a998-820cf48081f7 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:08.157216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457d191-5a6c-4c84-9437-454538fb0de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:08.157990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457d191-5a6c-4c84-9437-454538fb0de6 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:08.158269 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457d191-5a6c-4c84-9437-454538fb0de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:08.158540 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457d191-5a6c-4c84-9437-454538fb0de6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:08.168819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457d191-5a6c-4c84-9437-454538fb0de6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:08.170236 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457d191-5a6c-4c84-9437-454538fb0de6 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:08.187261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-b457d191-5a6c-4c84-9437-454538fb0de6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:03:08.187261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:08.187261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:08.187261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:08.187261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:08.187261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:08.187261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:08.187261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:08.187261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:08.187261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:08.187261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:08.187261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:08.187261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:03:08.187261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:03:08.187261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:08.188319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:08.189262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:03:08.189262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:03:08.189262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:03:08.189262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:03:08.189262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 634/1269] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:08.194133 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30979c67-cc4a-4291-acb5-c2cd38b2c9a1 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:08.297551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30979c67-cc4a-4291-acb5-c2cd38b2c9a1 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:08.336685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d745c96-5834-4e87-b0b6-393558254ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:08.337897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d745c96-5834-4e87-b0b6-393558254ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:08.338181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d745c96-5834-4e87-b0b6-393558254ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:08.338462 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d745c96-5834-4e87-b0b6-393558254ac6 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:08.339214 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d745c96-5834-4e87-b0b6-393558254ac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:08.339526 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d745c96-5834-4e87-b0b6-393558254ac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:08.340889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d745c96-5834-4e87-b0b6-393558254ac6 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:08.351952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 636/1270] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:08.357557 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457d191-5a6c-4c84-9437-454538fb0de6 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:08.390407 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:08.457261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457d191-5a6c-4c84-9437-454538fb0de6 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:08.489771 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5005694-d95b-4f99-8bdd-1e2c39b788aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:08.490548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5005694-d95b-4f99-8bdd-1e2c39b788aa None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:08.490858 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5005694-d95b-4f99-8bdd-1e2c39b788aa None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:08.491137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5005694-d95b-4f99-8bdd-1e2c39b788aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:08.498256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5005694-d95b-4f99-8bdd-1e2c39b788aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:08.499704 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5005694-d95b-4f99-8bdd-1e2c39b788aa None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:08.513413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-e5005694-d95b-4f99-8bdd-1e2c39b788aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:03:08.513413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:08.513413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:08.513413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:08.513413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:08.513413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:08.513413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:08.513413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:08.513413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:08.513413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:08.513413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:08.513413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:08.513413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:03:08.513413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:03:08.513413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:08.514419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:08.515762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:03:08.515762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:03:08.515762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:03:08.515762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:03:08.515762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:03:08.515762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:03:08.515762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:03:08.515762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:03:08.515762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 635/1271] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:03:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:08.520398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d745c96-5834-4e87-b0b6-393558254ac6 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:08.578381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:08.620131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d745c96-5834-4e87-b0b6-393558254ac6 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:08.652027 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87d1d2-218b-4b41-bfc4-40d30d278d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:08.652619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87d1d2-218b-4b41-bfc4-40d30d278d25 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:08.652902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87d1d2-218b-4b41-bfc4-40d30d278d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:08.653180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87d1d2-218b-4b41-bfc4-40d30d278d25 None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:08.653958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87d1d2-218b-4b41-bfc4-40d30d278d25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:08.654267 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87d1d2-218b-4b41-bfc4-40d30d278d25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:08.655745 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87d1d2-218b-4b41-bfc4-40d30d278d25 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:08.667751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 637/1272] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:03:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:08.673659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5005694-d95b-4f99-8bdd-1e2c39b788aa None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:08.762071 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:08.770639 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5005694-d95b-4f99-8bdd-1e2c39b788aa None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:08.803101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315aabc8-2458-4249-88c6-e912620b88a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:08.803989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315aabc8-2458-4249-88c6-e912620b88a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:08.804267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315aabc8-2458-4249-88c6-e912620b88a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:08.804538 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315aabc8-2458-4249-88c6-e912620b88a3 None admin] PATH_INFO: `/v3/projects/a498284274454d25b1b118a88996e522/users/066f69a87f5f48c7b20c1e0fcf5c1107/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:08.807154 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315aabc8-2458-4249-88c6-e912620b88a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a498284274454d25b1b118a88996e522, user_id=066f69a87f5f48c7b20c1e0fcf5c1107, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:08.808564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315aabc8-2458-4249-88c6-e912620b88a3 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:08.821444 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 636/1273] 10.222.0.44 () {58 vars in 1439 bytes} [Wed Jul 20 09:03:08 2022] PUT /identity/v3/projects/a498284274454d25b1b118a88996e522/users/066f69a87f5f48c7b20c1e0fcf5c1107/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:03:10.827251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ed89b-4d4d-4670-800c-1c62b54c4f15 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:10.827753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ed89b-4d4d-4670-800c-1c62b54c4f15 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:10.827753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ed89b-4d4d-4670-800c-1c62b54c4f15 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:10.829093 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 638/1274] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:03:10.834215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6afff8-8183-4bd2-9638-7344832e8641 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:10.834607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6afff8-8183-4bd2-9638-7344832e8641 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:10.834804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6afff8-8183-4bd2-9638-7344832e8641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:10.851767 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-8e6afff8-8183-4bd2-9638-7344832e8641 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:03:10.853573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8e6afff8-8183-4bd2-9638-7344832e8641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:03:10.918180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 637/1275] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:03:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:03:10.941606 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ed89b-4d4d-4670-800c-1c62b54c4f15 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:11.039725 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ed89b-4d4d-4670-800c-1c62b54c4f15 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:11.057462 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:11.074395 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e55fa-75ef-4994-89ee-bf8f1b81cb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:11.075198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e55fa-75ef-4994-89ee-bf8f1b81cb94 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:11.075487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e55fa-75ef-4994-89ee-bf8f1b81cb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:11.075759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e55fa-75ef-4994-89ee-bf8f1b81cb94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:11.085569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e55fa-75ef-4994-89ee-bf8f1b81cb94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 09:03:11.093115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-e18e55fa-75ef-4994-89ee-bf8f1b81cb94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:03:11.093115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:11.093115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:11.093115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:11.093115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:11.093115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:11.093115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:11.093115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:11.093115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:11.093115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:11.093115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:11.093115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:11.093115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 09:03:11.093115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 09:03:11.093115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 09:03:11.093115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 09:03:11.094239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 09:03:11.094239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 09:03:11.094239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 09:03:11.094239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 09:03:11.094239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:11.094239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:11.094239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:11.094239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:11.094239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:11.094239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:11.094239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:11.094239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:11.094239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:11.094239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:11.094239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:11.094239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:11.095173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:11.095173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:11.095173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:11.095173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:11.095173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 09:03:11.095173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 09:03:11.095173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 09:03:11.095173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 09:03:11.095173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 09:03:11.095173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 09:03:11.095173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 09:03:11.095173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:03:11.095173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 639/1276] 10.222.0.44 () {56 vars in 1096 bytes} [Wed Jul 20 09:03:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:11.100219 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6afff8-8183-4bd2-9638-7344832e8641 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:11.180194 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:11.199515 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6afff8-8183-4bd2-9638-7344832e8641 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:11.233048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccf84b-26f2-4819-a085-0fdda16b3fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:11.233707 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccf84b-26f2-4819-a085-0fdda16b3fee None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:11.233993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccf84b-26f2-4819-a085-0fdda16b3fee None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:11.234271 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ccf84b-26f2-4819-a085-0fdda16b3fee None admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:11.235048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccf84b-26f2-4819-a085-0fdda16b3fee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:11.235360 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccf84b-26f2-4819-a085-0fdda16b3fee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:11.236716 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ccf84b-26f2-4819-a085-0fdda16b3fee None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:11.244855 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 638/1277] 10.222.0.44 () {56 vars in 1099 bytes} [Wed Jul 20 09:03:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:11.250445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e55fa-75ef-4994-89ee-bf8f1b81cb94 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:11.367275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e55fa-75ef-4994-89ee-bf8f1b81cb94 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:11.399861 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbaa11-d8da-48b3-a5f7-9bb26b76b136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:11.400648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbaa11-d8da-48b3-a5f7-9bb26b76b136 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:11.400923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbaa11-d8da-48b3-a5f7-9bb26b76b136 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:11.401195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbaa11-d8da-48b3-a5f7-9bb26b76b136 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:11.411338 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbaa11-d8da-48b3-a5f7-9bb26b76b136 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:11.412853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbaa11-d8da-48b3-a5f7-9bb26b76b136 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:11.429946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-87cbaa11-d8da-48b3-a5f7-9bb26b76b136 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:03:11.429946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:11.429946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:11.429946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:11.429946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:11.429946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:11.429946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:11.429946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:11.429946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:11.429946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:11.429946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:11.429946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:11.429946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:03:11.429946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:03:11.429946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:11.430972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:11.431888 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:03:11.431888 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:03:11.431888 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:03:11.431888 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:03:11.431888 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 640/1278] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:11.437117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccf84b-26f2-4819-a085-0fdda16b3fee None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:11.536689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ccf84b-26f2-4819-a085-0fdda16b3fee None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:11.569084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3a46f-6723-4ead-99b9-ae9397578e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:11.569742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3a46f-6723-4ead-99b9-ae9397578e21 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:11.570029 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3a46f-6723-4ead-99b9-ae9397578e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:11.570573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3a46f-6723-4ead-99b9-ae9397578e21 None admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:11.572442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3a46f-6723-4ead-99b9-ae9397578e21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:11.572442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3a46f-6723-4ead-99b9-ae9397578e21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:11.573709 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3a46f-6723-4ead-99b9-ae9397578e21 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:11.585132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 639/1279] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:11.591710 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbaa11-d8da-48b3-a5f7-9bb26b76b136 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:11.622335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:11.691309 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbaa11-d8da-48b3-a5f7-9bb26b76b136 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:11.724707 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b370bfc-1af6-4e9a-a8ba-d64447f228d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:11.725492 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b370bfc-1af6-4e9a-a8ba-d64447f228d1 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:11.725778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b370bfc-1af6-4e9a-a8ba-d64447f228d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:11.726058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b370bfc-1af6-4e9a-a8ba-d64447f228d1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:11.733369 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b370bfc-1af6-4e9a-a8ba-d64447f228d1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:11.734831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b370bfc-1af6-4e9a-a8ba-d64447f228d1 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:11.748685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-3b370bfc-1af6-4e9a-a8ba-d64447f228d1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:03:11.748685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:11.748685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:11.748685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:11.748685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:11.748685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:11.748685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:11.748685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:11.748685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:11.748685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:11.748685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:11.748685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:11.748685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 09:03:11.748685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 09:03:11.748685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 09:03:11.748685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:11.749904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:11.751277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:11.751277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:11.751277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 09:03:11.751277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 09:03:11.751277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 09:03:11.751277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 09:03:11.751277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 09:03:11.751277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 09:03:11.751277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 09:03:11.751277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:03:11.751277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 641/1280] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:03:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:03:11.755665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3a46f-6723-4ead-99b9-ae9397578e21 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:11.759979 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:11.851467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3a46f-6723-4ead-99b9-ae9397578e21 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:11.884809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdcd49-613d-4903-b37c-6ebca2556c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:11.885405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdcd49-613d-4903-b37c-6ebca2556c6f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:11.885684 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdcd49-613d-4903-b37c-6ebca2556c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:11.885955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdcd49-613d-4903-b37c-6ebca2556c6f None admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:11.886805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdcd49-613d-4903-b37c-6ebca2556c6f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:11.887111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdcd49-613d-4903-b37c-6ebca2556c6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:11.888521 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdcd49-613d-4903-b37c-6ebca2556c6f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:11.904842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 640/1281] 10.222.0.44 () {56 vars in 1143 bytes} [Wed Jul 20 09:03:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:11.910433 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b370bfc-1af6-4e9a-a8ba-d64447f228d1 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:11.998490 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:12.008315 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b370bfc-1af6-4e9a-a8ba-d64447f228d1 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:12.041076 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af33dfe-5bba-445f-aea7-6a126f031200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:12.041705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af33dfe-5bba-445f-aea7-6a126f031200 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:12.041989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af33dfe-5bba-445f-aea7-6a126f031200 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:12.042267 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af33dfe-5bba-445f-aea7-6a126f031200 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:12.107401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af33dfe-5bba-445f-aea7-6a126f031200 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:12.108832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af33dfe-5bba-445f-aea7-6a126f031200 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:12.193560 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:12.204451 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 642/1282] 10.222.0.44 () {58 vars in 1282 bytes} [Wed Jul 20 09:03:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 09:03:12.210100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdcd49-613d-4903-b37c-6ebca2556c6f None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:12.269541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:12.307480 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdcd49-613d-4903-b37c-6ebca2556c6f None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:12.341022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955e03c-9b7a-400e-b900-642cfa67880e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:12.341800 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955e03c-9b7a-400e-b900-642cfa67880e None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:12.342080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955e03c-9b7a-400e-b900-642cfa67880e None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:12.342355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955e03c-9b7a-400e-b900-642cfa67880e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:12.352023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955e03c-9b7a-400e-b900-642cfa67880e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:12.354030 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955e03c-9b7a-400e-b900-642cfa67880e None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:12.372247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-b955e03c-9b7a-400e-b900-642cfa67880e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:03:12.372247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:12.372247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:12.372247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:12.372247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:12.372247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:12.372247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:12.372247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:12.372247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:12.372247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:12.372247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:12.372247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:12.372247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 09:03:12.372247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 09:03:12.372247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:12.373274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:12.374229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 09:03:12.374229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 09:03:12.374229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 09:03:12.374229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:03:12.374229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 641/1283] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 09:03:12.379085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af33dfe-5bba-445f-aea7-6a126f031200 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:12.475313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af33dfe-5bba-445f-aea7-6a126f031200 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:12.508694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b1100-1e29-4109-8929-4e2e6c8c4170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:12.509353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b1100-1e29-4109-8929-4e2e6c8c4170 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:12.509640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b1100-1e29-4109-8929-4e2e6c8c4170 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:12.509922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b1100-1e29-4109-8929-4e2e6c8c4170 None admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:12.510641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b1100-1e29-4109-8929-4e2e6c8c4170 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:12.510983 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b1100-1e29-4109-8929-4e2e6c8c4170 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:12.512345 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b1100-1e29-4109-8929-4e2e6c8c4170 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:12.523146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 643/1284] 10.222.0.44 () {56 vars in 1109 bytes} [Wed Jul 20 09:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:12.529019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955e03c-9b7a-400e-b900-642cfa67880e None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:12.626944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955e03c-9b7a-400e-b900-642cfa67880e None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:12.640387 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:12.660276 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79913b-7105-475b-9d77-7079d1685ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:12.661056 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79913b-7105-475b-9d77-7079d1685ace None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:12.661338 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79913b-7105-475b-9d77-7079d1685ace None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:12.661613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79913b-7105-475b-9d77-7079d1685ace None admin] PATH_INFO: `/v3/projects/service` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:12.668785 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79913b-7105-475b-9d77-7079d1685ace None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:12.670225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79913b-7105-475b-9d77-7079d1685ace None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:12.683699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application [None req-ad79913b-7105-475b-9d77-7079d1685ace None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:03:12.683699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:12.683699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:12.683699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:12.683699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:12.683699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:12.683699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:12.683699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:12.683699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:12.683699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:12.683699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:12.683699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:12.683699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 09:03:12.683699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 09:03:12.683699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:12.684770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:12.685723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 09:03:12.685723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 09:03:12.685723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 09:03:12.685723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 09:03:12.685723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 09:03:12.685723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 09:03:12.685723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 09:03:12.685723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: ERROR keystone.server.flask.application Jul 20 09:03:12.685723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 642/1285] 10.222.0.44 () {56 vars in 1111 bytes} [Wed Jul 20 09:03:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:12.690703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b1100-1e29-4109-8929-4e2e6c8c4170 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:12.739624 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:12.787065 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b1100-1e29-4109-8929-4e2e6c8c4170 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:12.819306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0fbd9-b244-4cd5-9c7b-ff032ab35a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:12.819946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0fbd9-b244-4cd5-9c7b-ff032ab35a59 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:12.820479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0fbd9-b244-4cd5-9c7b-ff032ab35a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:12.820754 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0fbd9-b244-4cd5-9c7b-ff032ab35a59 None admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:12.821654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0fbd9-b244-4cd5-9c7b-ff032ab35a59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:12.821961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0fbd9-b244-4cd5-9c7b-ff032ab35a59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:12.823402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0fbd9-b244-4cd5-9c7b-ff032ab35a59 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:12.834824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 644/1286] 10.222.0.44 () {56 vars in 1148 bytes} [Wed Jul 20 09:03:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:12.841321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79913b-7105-475b-9d77-7079d1685ace None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:12.939312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79913b-7105-475b-9d77-7079d1685ace None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:12.972168 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd855c8-9992-46d6-b281-540538be3405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:12.972810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd855c8-9992-46d6-b281-540538be3405 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:12.973095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd855c8-9992-46d6-b281-540538be3405 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:12.973382 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd855c8-9992-46d6-b281-540538be3405 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:12.974208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd855c8-9992-46d6-b281-540538be3405 None admin] RBAC: Adding query filter params (role.id=41447ccf04114fd2b491c81d6f43fd8c, scope.project.id=a498284274454d25b1b118a88996e522, user.id=066f69a87f5f48c7b20c1e0fcf5c1107) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:12.974525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd855c8-9992-46d6-b281-540538be3405 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:12.975964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd855c8-9992-46d6-b281-540538be3405 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:12.982310 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 643/1287] 10.222.0.44 () {56 vars in 1374 bytes} [Wed Jul 20 09:03:12 2022] GET /identity/v3/role_assignments?user.id=066f69a87f5f48c7b20c1e0fcf5c1107&scope.project.id=a498284274454d25b1b118a88996e522&role.id=41447ccf04114fd2b491c81d6f43fd8c => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:15.138423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6979ed-852b-418e-95be-459ad2f6afad None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:15.139037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6979ed-852b-418e-95be-459ad2f6afad None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:15.139037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6979ed-852b-418e-95be-459ad2f6afad None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:15.140362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 645/1288] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:03:15.145473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf5aae-4085-43d8-baa2-6aa3341eee0b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:15.145903 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf5aae-4085-43d8-baa2-6aa3341eee0b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:15.146030 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf5aae-4085-43d8-baa2-6aa3341eee0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:15.162298 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-13bf5aae-4085-43d8-baa2-6aa3341eee0b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:03:15.163515 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-13bf5aae-4085-43d8-baa2-6aa3341eee0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:03:15.192215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:15.229707 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 644/1289] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:03:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:03:15.251715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6979ed-852b-418e-95be-459ad2f6afad None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:15.356204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6979ed-852b-418e-95be-459ad2f6afad None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:15.367867 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:15.391613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819f090-1b22-445c-872a-84814842a653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:15.392338 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819f090-1b22-445c-872a-84814842a653 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:15.392616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819f090-1b22-445c-872a-84814842a653 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:15.392890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819f090-1b22-445c-872a-84814842a653 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:15.393536 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819f090-1b22-445c-872a-84814842a653 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:15.394926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819f090-1b22-445c-872a-84814842a653 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-2819f090-1b22-445c-872a-84814842a653 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:15.406890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:15.408137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:15.408137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:15.408137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:15.408137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:15.408137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:15.408137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:15.408137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:03:15.408137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:03:15.408137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:03:15.408137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:03:15.408137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:15.408137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:15.408137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:15.408137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:15.408137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:15.408137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:15.409317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:15.409317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:15.409317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:15.409317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:15.409317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:15.409317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:15.409317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:15.409317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:15.409317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:15.409317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:15.409317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:03:15.409317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:03:15.409317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:03:15.409317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:03:15.409317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 646/1290] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jul 20 09:03:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:15.414028 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf5aae-4085-43d8-baa2-6aa3341eee0b None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:15.509605 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf5aae-4085-43d8-baa2-6aa3341eee0b None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:15.542420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979c788-a3a0-436c-8a05-8ed5ae533312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:15.543109 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979c788-a3a0-436c-8a05-8ed5ae533312 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:15.543396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979c788-a3a0-436c-8a05-8ed5ae533312 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:15.543675 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979c788-a3a0-436c-8a05-8ed5ae533312 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:15.544384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979c788-a3a0-436c-8a05-8ed5ae533312 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:15.544691 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979c788-a3a0-436c-8a05-8ed5ae533312 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:15.546048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979c788-a3a0-436c-8a05-8ed5ae533312 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:15.550916 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 645/1291] 10.222.0.44 () {56 vars in 1116 bytes} [Wed Jul 20 09:03:15 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:15.557215 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819f090-1b22-445c-872a-84814842a653 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:15.652896 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819f090-1b22-445c-872a-84814842a653 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:15.684927 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94925a-0d41-4807-a465-2ae46824041e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:15.685563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94925a-0d41-4807-a465-2ae46824041e None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:15.685844 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94925a-0d41-4807-a465-2ae46824041e None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:15.686118 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94925a-0d41-4807-a465-2ae46824041e None admin] PATH_INFO: `/v3/services` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:15.686837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94925a-0d41-4807-a465-2ae46824041e None admin] RBAC: Adding query filter params (type=placement) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:15.687140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94925a-0d41-4807-a465-2ae46824041e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:15.688487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94925a-0d41-4807-a465-2ae46824041e None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:15.692870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 647/1292] 10.222.0.44 () {56 vars in 1116 bytes} [Wed Jul 20 09:03:15 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:03:17.671085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40c399-b562-4aef-8e61-3a08b504c300 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:17.671619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40c399-b562-4aef-8e61-3a08b504c300 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:17.671619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40c399-b562-4aef-8e61-3a08b504c300 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:17.673000 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 646/1293] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:03:17.678011 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e1624-fe1f-4f64-b6a5-26074111857d None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:17.678391 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e1624-fe1f-4f64-b6a5-26074111857d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:17.678574 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e1624-fe1f-4f64-b6a5-26074111857d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:17.695595 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-e88e1624-fe1f-4f64-b6a5-26074111857d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:03:17.696789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e88e1624-fe1f-4f64-b6a5-26074111857d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:03:17.724607 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:17.761385 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 648/1294] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:03:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:03:17.783722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40c399-b562-4aef-8e61-3a08b504c300 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:17.815309 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:17.885520 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40c399-b562-4aef-8e61-3a08b504c300 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:17.919316 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9547f2-6d79-4ace-921b-fb932b0c6997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:17.919972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9547f2-6d79-4ace-921b-fb932b0c6997 None admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:17.920259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9547f2-6d79-4ace-921b-fb932b0c6997 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:17.920548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9547f2-6d79-4ace-921b-fb932b0c6997 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:17.921271 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9547f2-6d79-4ace-921b-fb932b0c6997 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:17.922635 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9547f2-6d79-4ace-921b-fb932b0c6997 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:17.935335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 647/1295] 10.222.0.44 () {60 vars in 1141 bytes} [Wed Jul 20 09:03:17 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:03:19.969065 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af11cc-213d-442f-b5e2-c079ca734518 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:19.969626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af11cc-213d-442f-b5e2-c079ca734518 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:19.969626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af11cc-213d-442f-b5e2-c079ca734518 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:19.971013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 649/1296] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:03:19.976122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d8b71-f7cd-4a6e-8b4d-0c64f2e08d9b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:19.976549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d8b71-f7cd-4a6e-8b4d-0c64f2e08d9b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:19.976691 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d8b71-f7cd-4a6e-8b4d-0c64f2e08d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:19.994455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-ca4d8b71-f7cd-4a6e-8b4d-0c64f2e08d9b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:03:19.995682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ca4d8b71-f7cd-4a6e-8b4d-0c64f2e08d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:03:20.066045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 648/1297] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:03:19 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:03:20.089367 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af11cc-213d-442f-b5e2-c079ca734518 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:20.187834 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af11cc-213d-442f-b5e2-c079ca734518 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:20.221300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a721f-620c-446f-883f-f11d67327fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:20.222019 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a721f-620c-446f-883f-f11d67327fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:20.222298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a721f-620c-446f-883f-f11d67327fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:20.222572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a721f-620c-446f-883f-f11d67327fe2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:20.223248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a721f-620c-446f-883f-f11d67327fe2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:20.224596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a721f-620c-446f-883f-f11d67327fe2 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-a11a721f-620c-446f-883f-f11d67327fe2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:20.236720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:20.237908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:20.237908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:20.237908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:20.237908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:20.237908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:20.237908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:20.237908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:03:20.237908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:03:20.237908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:03:20.237908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:03:20.237908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:20.237908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:20.237908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:20.237908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:20.237908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:20.237908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:20.239099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:20.239099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:20.239099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:20.239099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:20.239099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:20.239099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:20.239099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:20.239099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:20.239099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:20.239099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:20.239099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:03:20.239099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:03:20.239099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:03:20.239099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:03:20.239099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 650/1298] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jul 20 09:03:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:20.243862 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d8b71-f7cd-4a6e-8b4d-0c64f2e08d9b None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:20.261293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:20.341956 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d8b71-f7cd-4a6e-8b4d-0c64f2e08d9b None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:20.374597 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd9e60-c296-4fce-8197-6ec8ca561a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:20.375275 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd9e60-c296-4fce-8197-6ec8ca561a64 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:20.375555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd9e60-c296-4fce-8197-6ec8ca561a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:20.375828 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd9e60-c296-4fce-8197-6ec8ca561a64 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:20.376524 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd9e60-c296-4fce-8197-6ec8ca561a64 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:20.376825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd9e60-c296-4fce-8197-6ec8ca561a64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:20.378168 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd9e60-c296-4fce-8197-6ec8ca561a64 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:20.383151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 649/1299] 10.222.0.44 () {56 vars in 1116 bytes} [Wed Jul 20 09:03:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:03:20.389237 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a721f-620c-446f-883f-f11d67327fe2 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:20.420828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:20.489793 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a721f-620c-446f-883f-f11d67327fe2 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:20.522402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21de4c-44cc-4af1-b455-0504084ef7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:20.523102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21de4c-44cc-4af1-b455-0504084ef7fb None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:20.523385 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21de4c-44cc-4af1-b455-0504084ef7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:20.523660 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21de4c-44cc-4af1-b455-0504084ef7fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:20.525256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21de4c-44cc-4af1-b455-0504084ef7fb None admin] RBAC: Adding query filter params (interface=public, service_id=018488bf55d74b508c9988f51d6871f4, region_id=RegionOne) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:20.525570 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21de4c-44cc-4af1-b455-0504084ef7fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:20.526944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21de4c-44cc-4af1-b455-0504084ef7fb None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:20.531850 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 651/1300] 10.222.0.44 () {56 vars in 1251 bytes} [Wed Jul 20 09:03:20 2022] GET /identity/v3/endpoints?service_id=018488bf55d74b508c9988f51d6871f4&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:20.537663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd9e60-c296-4fce-8197-6ec8ca561a64 None admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:20.618017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:20.636890 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd9e60-c296-4fce-8197-6ec8ca561a64 None admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:20.669972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66651fa-5ea1-4352-9783-ca1186af490f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:20.670628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66651fa-5ea1-4352-9783-ca1186af490f None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:20.670941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66651fa-5ea1-4352-9783-ca1186af490f None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:20.671220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66651fa-5ea1-4352-9783-ca1186af490f None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:20.671901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66651fa-5ea1-4352-9783-ca1186af490f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:20.673255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66651fa-5ea1-4352-9783-ca1186af490f None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:20.679918 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 650/1301] 10.222.0.44 () {56 vars in 1087 bytes} [Wed Jul 20 09:03:20 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:03:22.668409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0c69f-e8cb-430d-91b9-f120815607cc None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:22.668971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0c69f-e8cb-430d-91b9-f120815607cc None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:22.668971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0c69f-e8cb-430d-91b9-f120815607cc None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:22.670306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 652/1302] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:03:22.675304 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeca7a-cd69-41ab-8ae0-9fc3f024a987 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:22.675670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeca7a-cd69-41ab-8ae0-9fc3f024a987 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:22.675860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeca7a-cd69-41ab-8ae0-9fc3f024a987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:22.692510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-43aeca7a-cd69-41ab-8ae0-9fc3f024a987 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:03:22.693682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-43aeca7a-cd69-41ab-8ae0-9fc3f024a987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:03:22.756924 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 651/1303] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:03:22 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 09:03:22.779490 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0c69f-e8cb-430d-91b9-f120815607cc None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:22.872545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:22.879550 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0c69f-e8cb-430d-91b9-f120815607cc None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:22.912299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d4bec-cfc5-49a1-9446-e285b4922fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:22.913074 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d4bec-cfc5-49a1-9446-e285b4922fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:22.913349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d4bec-cfc5-49a1-9446-e285b4922fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:22.913617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d4bec-cfc5-49a1-9446-e285b4922fd9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:22.914253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d4bec-cfc5-49a1-9446-e285b4922fd9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:22.916247 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d4bec-cfc5-49a1-9446-e285b4922fd9 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application [None req-a24d4bec-cfc5-49a1-9446-e285b4922fd9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 09:03:22.927742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 09:03:22.928741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 09:03:22.928741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 09:03:22.928741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 09:03:22.928741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 09:03:22.928741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 09:03:22.928741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 09:03:22.928741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 09:03:22.928741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 09:03:22.928741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 09:03:22.928741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 09:03:22.928741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 09:03:22.928741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 09:03:22.928741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 09:03:22.928741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 09:03:22.928741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 09:03:22.928741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 09:03:22.929747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 09:03:22.929747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application with Lock( Jul 20 09:03:22.929747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 09:03:22.929747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self._enter() Jul 20 09:03:22.929747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 09:03:22.929747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 09:03:22.929747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 09:03:22.929747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application return self.creator() Jul 20 09:03:22.929747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 09:03:22.929747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application created_value = creator( Jul 20 09:03:22.929747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 09:03:22.929747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 09:03:22.929747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 09:03:22.929747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: ERROR keystone.server.flask.application Jul 20 09:03:22.929747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 653/1304] 10.222.0.44 () {56 vars in 1117 bytes} [Wed Jul 20 09:03:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 09:03:22.935624 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aeca7a-cd69-41ab-8ae0-9fc3f024a987 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:23.008941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:23.033139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aeca7a-cd69-41ab-8ae0-9fc3f024a987 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:23.066026 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9d6d0-f926-426f-8d82-55bac15124e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:23.066671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9d6d0-f926-426f-8d82-55bac15124e6 None admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:23.066980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9d6d0-f926-426f-8d82-55bac15124e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:23.067254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9d6d0-f926-426f-8d82-55bac15124e6 None admin] PATH_INFO: `/v3/services` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:23.067956 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9d6d0-f926-426f-8d82-55bac15124e6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:03:23.068261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9d6d0-f926-426f-8d82-55bac15124e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:23.069610 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9d6d0-f926-426f-8d82-55bac15124e6 None admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:23.075001 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 652/1305] 10.222.0.44 () {56 vars in 1116 bytes} [Wed Jul 20 09:03:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:03:23.080611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d4bec-cfc5-49a1-9446-e285b4922fd9 None admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:03:23.178617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d4bec-cfc5-49a1-9446-e285b4922fd9 None admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:03:23.211781 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf19bd0-d29f-47fa-acb3-c75f2ddb5496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:03:23.212439 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf19bd0-d29f-47fa-acb3-c75f2ddb5496 None admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:03:23.212722 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf19bd0-d29f-47fa-acb3-c75f2ddb5496 None admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:03:23.213002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf19bd0-d29f-47fa-acb3-c75f2ddb5496 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:03:23.213744 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf19bd0-d29f-47fa-acb3-c75f2ddb5496 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:03:23.215150 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf19bd0-d29f-47fa-acb3-c75f2ddb5496 None admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:03:23.233087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 654/1306] 10.222.0.44 () {60 vars in 1144 bytes} [Wed Jul 20 09:03:23 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:03:33.072007 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=30 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:03:33.229199 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:26.085469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff559c-d8f2-42a9-a289-3f21a157bb0f None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:26.086509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff559c-d8f2-42a9-a289-3f21a157bb0f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:26.087049 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff559c-d8f2-42a9-a289-3f21a157bb0f None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:26.092432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 653/1307] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:04:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:04:26.093858 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02f6df-3255-43ee-bffd-0e94e338a67e None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:26.094265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02f6df-3255-43ee-bffd-0e94e338a67e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:26.094430 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02f6df-3255-43ee-bffd-0e94e338a67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:26.171086 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-8c02f6df-3255-43ee-bffd-0e94e338a67e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:04:26.172286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8c02f6df-3255-43ee-bffd-0e94e338a67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:26.276974 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 655/1308] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:04:26.339826 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42e28c-e003-409e-bd9c-88791dbd9bba None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:26.340122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42e28c-e003-409e-bd9c-88791dbd9bba None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:26.340408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42e28c-e003-409e-bd9c-88791dbd9bba None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:26.342041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 654/1309] 10.222.0.44 () {54 vars in 965 bytes} [Wed Jul 20 09:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:04:26.348163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2e697-0d44-492e-83ac-4e7528d0ecda None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:26.348559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2e697-0d44-492e-83ac-4e7528d0ecda None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:26.348642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2e697-0d44-492e-83ac-4e7528d0ecda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:26.427364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-cdb2e697-0d44-492e-83ac-4e7528d0ecda None None] MFA Rules not processed for user `5e5ad4e122ae471cad4cf1b0e3111dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:04:26.428764 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-cdb2e697-0d44-492e-83ac-4e7528d0ecda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:26.457285 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:26.531092 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 656/1310] 10.222.0.44 () {58 vars in 1062 bytes} [Wed Jul 20 09:04:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:04:26.532171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42e28c-e003-409e-bd9c-88791dbd9bba None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:04:26.534208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7c42e28c-e003-409e-bd9c-88791dbd9bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:26.681781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c42e28c-e003-409e-bd9c-88791dbd9bba None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:04:26.770402 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafcb37-13b7-45aa-85e8-bbd16947fd29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:04:26.771091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafcb37-13b7-45aa-85e8-bbd16947fd29 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:26.771377 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafcb37-13b7-45aa-85e8-bbd16947fd29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:26.771659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafcb37-13b7-45aa-85e8-bbd16947fd29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:26.773767 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-beafcb37-13b7-45aa-85e8-bbd16947fd29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:26.831694 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:26.890018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beafcb37-13b7-45aa-85e8-bbd16947fd29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:04:26.891499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beafcb37-13b7-45aa-85e8-bbd16947fd29 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:04:26.911547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 655/1311] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:04:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:04:29.280498 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672a912-8841-4e03-bb78-32d77274175f None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:29.281058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672a912-8841-4e03-bb78-32d77274175f None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:29.281058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672a912-8841-4e03-bb78-32d77274175f None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:29.282778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 657/1312] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:04:29.288110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5aaa0-bcf9-4450-bff5-9a4bf31086ca None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:29.289094 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5aaa0-bcf9-4450-bff5-9a4bf31086ca None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:29.289329 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5aaa0-bcf9-4450-bff5-9a4bf31086ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:29.380769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-c3d5aaa0-bcf9-4450-bff5-9a4bf31086ca None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:04:29.382224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c3d5aaa0-bcf9-4450-bff5-9a4bf31086ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:29.485487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 656/1313] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:04:29.524493 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672a912-8841-4e03-bb78-32d77274175f None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:04:29.526004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8672a912-8841-4e03-bb78-32d77274175f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:29.651998 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672a912-8841-4e03-bb78-32d77274175f None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:04:29.731760 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:29.734969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8d3ac-2a1f-4287-aa19-1921ffea1376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:04:29.735705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8d3ac-2a1f-4287-aa19-1921ffea1376 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:29.736244 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8d3ac-2a1f-4287-aa19-1921ffea1376 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:29.736767 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8d3ac-2a1f-4287-aa19-1921ffea1376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:29.738908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ebe8d3ac-2a1f-4287-aa19-1921ffea1376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:29.847159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8d3ac-2a1f-4287-aa19-1921ffea1376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:04:29.848547 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8d3ac-2a1f-4287-aa19-1921ffea1376 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:04:29.867416 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 658/1314] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:04:32.197239 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bdff3-8a2f-4314-9b66-e41b0a817b65 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:32.197796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bdff3-8a2f-4314-9b66-e41b0a817b65 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:32.197796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bdff3-8a2f-4314-9b66-e41b0a817b65 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:32.199235 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 657/1315] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:04:32.204394 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dae48f-ed0c-48dd-90c6-91bbb4182d56 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:32.204800 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dae48f-ed0c-48dd-90c6-91bbb4182d56 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:32.204946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dae48f-ed0c-48dd-90c6-91bbb4182d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:32.268756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-83dae48f-ed0c-48dd-90c6-91bbb4182d56 None None] MFA Rules not processed for user `7e0b262e8410401e8d9297ca2732da45`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:04:32.269889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-83dae48f-ed0c-48dd-90c6-91bbb4182d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:32.322488 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:32.365029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 659/1316] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:04:32 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:04:32.404917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9bdff3-8a2f-4314-9b66-e41b0a817b65 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:04:32.406476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7e9bdff3-8a2f-4314-9b66-e41b0a817b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:32.416380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:32.511564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9bdff3-8a2f-4314-9b66-e41b0a817b65 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:04:32.584210 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7d962-a0a0-4830-86a0-fb44d94f5da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:04:32.585439 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7d962-a0a0-4830-86a0-fb44d94f5da7 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:32.585725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7d962-a0a0-4830-86a0-fb44d94f5da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:32.586005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7d962-a0a0-4830-86a0-fb44d94f5da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:32.587672 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-60e7d962-a0a0-4830-86a0-fb44d94f5da7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:32.648529 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:32.695095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e7d962-a0a0-4830-86a0-fb44d94f5da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:04:32.696788 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e7d962-a0a0-4830-86a0-fb44d94f5da7 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:04:32.715162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 658/1317] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:04:32 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:04:35.335505 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79bee6-1305-481d-bae1-3e16a3b34454 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:35.336041 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79bee6-1305-481d-bae1-3e16a3b34454 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:35.336041 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79bee6-1305-481d-bae1-3e16a3b34454 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:35.337438 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 660/1318] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:04:35.342848 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc75472-2088-4a27-a451-0a4fcba27a0d None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:35.343223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc75472-2088-4a27-a451-0a4fcba27a0d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:35.343417 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc75472-2088-4a27-a451-0a4fcba27a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:35.407611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-9bc75472-2088-4a27-a451-0a4fcba27a0d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:04:35.408811 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9bc75472-2088-4a27-a451-0a4fcba27a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:35.503122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 659/1319] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:04:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:04:35.542449 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79bee6-1305-481d-bae1-3e16a3b34454 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:04:35.544001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3d79bee6-1305-481d-bae1-3e16a3b34454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:35.645030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79bee6-1305-481d-bae1-3e16a3b34454 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:04:35.715928 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4ba87-ee16-4467-a375-8e3020cd325f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:04:35.716563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4ba87-ee16-4467-a375-8e3020cd325f service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:35.716843 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4ba87-ee16-4467-a375-8e3020cd325f service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:35.717121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4ba87-ee16-4467-a375-8e3020cd325f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:35.718771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-63b4ba87-ee16-4467-a375-8e3020cd325f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:35.726590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:35.824911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4ba87-ee16-4467-a375-8e3020cd325f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:04:35.826336 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4ba87-ee16-4467-a375-8e3020cd325f service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:04:35.844530 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 661/1320] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:04:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:04:38.565304 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbe6bd-358a-46d6-bb15-55da7343de20 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:38.565775 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbe6bd-358a-46d6-bb15-55da7343de20 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:38.565894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbe6bd-358a-46d6-bb15-55da7343de20 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:38.567329 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 660/1321] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:04:38.573518 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09ce55-f20c-419a-b5bf-1f3815124d70 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:38.573938 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09ce55-f20c-419a-b5bf-1f3815124d70 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:38.574088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09ce55-f20c-419a-b5bf-1f3815124d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:38.638897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-4b09ce55-f20c-419a-b5bf-1f3815124d70 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:04:38.640150 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4b09ce55-f20c-419a-b5bf-1f3815124d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:38.701485 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:38.746051 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 662/1322] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:04:38.797263 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd9627-f1fc-4c4d-b479-44763e45b641 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:38.797625 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd9627-f1fc-4c4d-b479-44763e45b641 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:38.797861 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd9627-f1fc-4c4d-b479-44763e45b641 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:38.799303 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 661/1323] 10.222.0.44 () {54 vars in 965 bytes} [Wed Jul 20 09:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:04:38.806013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a760b-0b86-4a4d-a067-87c17a505d43 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:38.806199 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a760b-0b86-4a4d-a067-87c17a505d43 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:38.806481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a760b-0b86-4a4d-a067-87c17a505d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:38.892434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-ca2a760b-0b86-4a4d-a067-87c17a505d43 None None] MFA Rules not processed for user `5e5ad4e122ae471cad4cf1b0e3111dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:04:38.893656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ca2a760b-0b86-4a4d-a067-87c17a505d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:38.972429 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 663/1324] 10.222.0.44 () {58 vars in 1062 bytes} [Wed Jul 20 09:04:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:04:38.982052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd9627-f1fc-4c4d-b479-44763e45b641 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:04:38.983728 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-edfd9627-f1fc-4c4d-b479-44763e45b641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:39.002745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:39.089080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd9627-f1fc-4c4d-b479-44763e45b641 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:04:39.161163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b104512-e3ba-488e-9f6e-07a1f0bcb138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:04:39.161802 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b104512-e3ba-488e-9f6e-07a1f0bcb138 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:39.162083 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b104512-e3ba-488e-9f6e-07a1f0bcb138 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:39.162474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b104512-e3ba-488e-9f6e-07a1f0bcb138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:39.164147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9b104512-e3ba-488e-9f6e-07a1f0bcb138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:39.236682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:39.276320 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b104512-e3ba-488e-9f6e-07a1f0bcb138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:04:39.277768 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b104512-e3ba-488e-9f6e-07a1f0bcb138 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:04:39.296040 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 662/1325] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:04:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:04:41.645232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a298069-506f-4248-91e6-1b206f79dce0 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:41.645762 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a298069-506f-4248-91e6-1b206f79dce0 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:41.645883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a298069-506f-4248-91e6-1b206f79dce0 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:41.647316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 664/1326] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:04:41.652604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70304fc-aa7c-4290-927a-025ca26e3819 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:41.653014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70304fc-aa7c-4290-927a-025ca26e3819 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:41.653200 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70304fc-aa7c-4290-927a-025ca26e3819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:41.718052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-d70304fc-aa7c-4290-927a-025ca26e3819 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:04:41.719289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d70304fc-aa7c-4290-927a-025ca26e3819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:41.825634 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 663/1327] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:04:41.862212 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a298069-506f-4248-91e6-1b206f79dce0 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:04:41.863809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-9a298069-506f-4248-91e6-1b206f79dce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:41.924592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:41.970151 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a298069-506f-4248-91e6-1b206f79dce0 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:04:42.043952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3b510-cab2-4b8d-80dc-dc5b974792ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:04:42.044529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3b510-cab2-4b8d-80dc-dc5b974792ae service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:42.044812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3b510-cab2-4b8d-80dc-dc5b974792ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:42.045089 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3b510-cab2-4b8d-80dc-dc5b974792ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:42.046716 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-82b3b510-cab2-4b8d-80dc-dc5b974792ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:42.153684 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:42.156027 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3b510-cab2-4b8d-80dc-dc5b974792ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:04:42.157764 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3b510-cab2-4b8d-80dc-dc5b974792ae service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:04:42.176842 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 665/1328] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:04:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:04:44.285507 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd8373-d4fa-4e2f-bd97-04843699df0a None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:44.286023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd8373-d4fa-4e2f-bd97-04843699df0a None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:44.286123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd8373-d4fa-4e2f-bd97-04843699df0a None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:44.288164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 664/1329] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:04:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:04:44.293458 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc358f83-0dd2-4e0c-afae-62f19826731f None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:44.293849 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc358f83-0dd2-4e0c-afae-62f19826731f None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:44.294041 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc358f83-0dd2-4e0c-afae-62f19826731f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:44.370416 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-bc358f83-0dd2-4e0c-afae-62f19826731f None None] MFA Rules not processed for user `7e0b262e8410401e8d9297ca2732da45`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:04:44.371579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-bc358f83-0dd2-4e0c-afae-62f19826731f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:44.475470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 666/1330] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:04:44 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:04:44.513663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd8373-d4fa-4e2f-bd97-04843699df0a None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:04:44.515258 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-54bd8373-d4fa-4e2f-bd97-04843699df0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:44.547163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:44.624998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd8373-d4fa-4e2f-bd97-04843699df0a None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:04:44.699531 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c7e0b-bb0f-41c9-9ef8-8d7085df67b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:04:44.700198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c7e0b-bb0f-41c9-9ef8-8d7085df67b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:44.700510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c7e0b-bb0f-41c9-9ef8-8d7085df67b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:44.700790 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c7e0b-bb0f-41c9-9ef8-8d7085df67b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:44.702449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-cb6c7e0b-bb0f-41c9-9ef8-8d7085df67b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:44.781547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:44.813022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c7e0b-bb0f-41c9-9ef8-8d7085df67b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:04:44.814448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c7e0b-bb0f-41c9-9ef8-8d7085df67b2 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:04:44.833437 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 665/1331] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:04:44 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:04:47.304167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb62f2c-175a-4c61-b59d-f6be6740f308 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:47.304812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb62f2c-175a-4c61-b59d-f6be6740f308 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:47.304812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb62f2c-175a-4c61-b59d-f6be6740f308 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:47.306104 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 667/1332] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:04:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:04:47.311306 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db7446-ab1a-481c-8ba8-c3d424f041b6 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:47.311755 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db7446-ab1a-481c-8ba8-c3d424f041b6 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:47.311858 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db7446-ab1a-481c-8ba8-c3d424f041b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:47.375925 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-88db7446-ab1a-481c-8ba8-c3d424f041b6 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:04:47.377157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-88db7446-ab1a-481c-8ba8-c3d424f041b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:47.470436 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 666/1333] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:04:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:04:47.507124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb62f2c-175a-4c61-b59d-f6be6740f308 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:04:47.508643 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-adb62f2c-175a-4c61-b59d-f6be6740f308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:47.579780 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:47.613780 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb62f2c-175a-4c61-b59d-f6be6740f308 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:04:47.685899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f1c32-11d8-4736-b4e5-9b599c9ad859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:04:47.686535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f1c32-11d8-4736-b4e5-9b599c9ad859 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:47.686846 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f1c32-11d8-4736-b4e5-9b599c9ad859 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:47.687122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f1c32-11d8-4736-b4e5-9b599c9ad859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:47.688729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-9f3f1c32-11d8-4736-b4e5-9b599c9ad859 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:47.795323 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f1c32-11d8-4736-b4e5-9b599c9ad859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:04:47.796773 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f1c32-11d8-4736-b4e5-9b599c9ad859 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:04:47.807117 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:47.816543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 668/1334] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:04:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:04:49.877760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ed0bc-3cd1-4d09-a1db-bba45be2ef28 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:49.878299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ed0bc-3cd1-4d09-a1db-bba45be2ef28 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:49.878404 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ed0bc-3cd1-4d09-a1db-bba45be2ef28 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:49.879756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 667/1335] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:04:49.884927 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6bb382-8613-4fbb-a8b4-3b8cbb2b1282 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:49.885350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6bb382-8613-4fbb-a8b4-3b8cbb2b1282 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:49.885469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6bb382-8613-4fbb-a8b4-3b8cbb2b1282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:49.948704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-8a6bb382-8613-4fbb-a8b4-3b8cbb2b1282 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:04:49.949899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8a6bb382-8613-4fbb-a8b4-3b8cbb2b1282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:50.044508 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 669/1336] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:04:50.085788 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ed0bc-3cd1-4d09-a1db-bba45be2ef28 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:04:50.087325 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ae4ed0bc-3cd1-4d09-a1db-bba45be2ef28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:50.127427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:50.190538 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ed0bc-3cd1-4d09-a1db-bba45be2ef28 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:04:50.263421 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f001885-cc7e-4540-ae56-f6d730060e98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:04:50.264071 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f001885-cc7e-4540-ae56-f6d730060e98 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:50.264367 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f001885-cc7e-4540-ae56-f6d730060e98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:50.264648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f001885-cc7e-4540-ae56-f6d730060e98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:50.266280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9f001885-cc7e-4540-ae56-f6d730060e98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:50.351224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:50.377058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f001885-cc7e-4540-ae56-f6d730060e98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:04:50.378498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f001885-cc7e-4540-ae56-f6d730060e98 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:04:50.397149 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 668/1337] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:04:52.875778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f559b2a-4c12-4885-b276-ca2021f4d4f7 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:52.876333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f559b2a-4c12-4885-b276-ca2021f4d4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:52.876333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f559b2a-4c12-4885-b276-ca2021f4d4f7 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:52.877702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 670/1338] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:04:52.883605 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c720e14-3a71-4fd0-9329-b920e23660ef None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:52.884037 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c720e14-3a71-4fd0-9329-b920e23660ef None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:52.884170 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c720e14-3a71-4fd0-9329-b920e23660ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:52.949212 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-1c720e14-3a71-4fd0-9329-b920e23660ef None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:04:52.950428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-1c720e14-3a71-4fd0-9329-b920e23660ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:53.044244 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 669/1339] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:04:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:04:53.081203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f559b2a-4c12-4885-b276-ca2021f4d4f7 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:04:53.082772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0f559b2a-4c12-4885-b276-ca2021f4d4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:53.158669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:53.187274 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f559b2a-4c12-4885-b276-ca2021f4d4f7 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:04:53.260031 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402899a-7eaf-476e-8fce-badb4a072be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:04:53.260679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402899a-7eaf-476e-8fce-badb4a072be7 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:53.260967 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402899a-7eaf-476e-8fce-badb4a072be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:53.261245 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402899a-7eaf-476e-8fce-badb4a072be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:53.262887 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-2402899a-7eaf-476e-8fce-badb4a072be7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:53.370157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402899a-7eaf-476e-8fce-badb4a072be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:04:53.371622 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2402899a-7eaf-476e-8fce-badb4a072be7 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:04:53.390264 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 671/1340] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:04:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:04:57.375527 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffbe4b-a0fd-4038-bdc7-a10c8c6d6f9e None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:57.376067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffbe4b-a0fd-4038-bdc7-a10c8c6d6f9e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:57.376067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ffbe4b-a0fd-4038-bdc7-a10c8c6d6f9e None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:57.377483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 670/1341] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:04:57.383596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301c635-83d6-4a28-9ba2-fb7b79b2a058 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:57.384031 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301c635-83d6-4a28-9ba2-fb7b79b2a058 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:57.384295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301c635-83d6-4a28-9ba2-fb7b79b2a058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:57.388557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:57.451009 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-7301c635-83d6-4a28-9ba2-fb7b79b2a058 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:04:57.452220 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-7301c635-83d6-4a28-9ba2-fb7b79b2a058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:57.549718 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 672/1342] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:04:57.589616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ffbe4b-a0fd-4038-bdc7-a10c8c6d6f9e None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:04:57.591218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-85ffbe4b-a0fd-4038-bdc7-a10c8c6d6f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:57.639894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:57.696108 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ffbe4b-a0fd-4038-bdc7-a10c8c6d6f9e None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:04:57.774294 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0b5dd-0396-4210-95e1-7c7dfed08d81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:04:57.774980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0b5dd-0396-4210-95e1-7c7dfed08d81 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:04:57.775271 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0b5dd-0396-4210-95e1-7c7dfed08d81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:04:57.775550 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0b5dd-0396-4210-95e1-7c7dfed08d81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:04:57.777173 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-89b0b5dd-0396-4210-95e1-7c7dfed08d81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:04:57.869875 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:04:57.887016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0b5dd-0396-4210-95e1-7c7dfed08d81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:04:57.888461 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0b5dd-0396-4210-95e1-7c7dfed08d81 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:04:57.907105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 671/1343] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:04:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:05:00.531494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48da3d-4a46-4184-adb4-e85b7fb555df None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:00.532050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48da3d-4a46-4184-adb4-e85b7fb555df None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:00.532050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48da3d-4a46-4184-adb4-e85b7fb555df None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:00.533632 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 673/1344] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:05:00.539442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f424cc7-5615-49f0-a579-249ff160568e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:00.539957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f424cc7-5615-49f0-a579-249ff160568e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:00.540035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f424cc7-5615-49f0-a579-249ff160568e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:00.619096 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-1f424cc7-5615-49f0-a579-249ff160568e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:00.620371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-1f424cc7-5615-49f0-a579-249ff160568e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:00.724695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 672/1345] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:05:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:00.763546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48da3d-4a46-4184-adb4-e85b7fb555df None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:00.765098 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-eb48da3d-4a46-4184-adb4-e85b7fb555df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:00.852256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:00.870245 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48da3d-4a46-4184-adb4-e85b7fb555df None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:00.945205 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd9bd2-5f45-4469-beda-9fde2b9fbe94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:00.946042 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd9bd2-5f45-4469-beda-9fde2b9fbe94 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:00.946329 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd9bd2-5f45-4469-beda-9fde2b9fbe94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:00.946613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd9bd2-5f45-4469-beda-9fde2b9fbe94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:00.948341 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-95dd9bd2-5f45-4469-beda-9fde2b9fbe94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:01.059313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd9bd2-5f45-4469-beda-9fde2b9fbe94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:01.060762 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd9bd2-5f45-4469-beda-9fde2b9fbe94 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:01.080270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 674/1346] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:05:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:05:05.387062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23260a6-b229-49ea-a08c-841cab25179f None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:05.387567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23260a6-b229-49ea-a08c-841cab25179f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:05.387679 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23260a6-b229-49ea-a08c-841cab25179f None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:05.388993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 673/1347] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:05.394626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7464e10-7efe-4a48-9782-e05ba378a14c None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:05.395076 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7464e10-7efe-4a48-9782-e05ba378a14c None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:05.395246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7464e10-7efe-4a48-9782-e05ba378a14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:05.409700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:05.479536 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-c7464e10-7efe-4a48-9782-e05ba378a14c None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:05.480751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c7464e10-7efe-4a48-9782-e05ba378a14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:05.589322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 675/1348] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:05.628474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23260a6-b229-49ea-a08c-841cab25179f None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:05.630025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-a23260a6-b229-49ea-a08c-841cab25179f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:05.691602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:05.741315 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23260a6-b229-49ea-a08c-841cab25179f None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:05.818135 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d393f-16db-42a2-9cb8-ce6ee0a41b59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:05.819065 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d393f-16db-42a2-9cb8-ce6ee0a41b59 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:05.819351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d393f-16db-42a2-9cb8-ce6ee0a41b59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:05.819629 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d393f-16db-42a2-9cb8-ce6ee0a41b59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:05.821555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-209d393f-16db-42a2-9cb8-ce6ee0a41b59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:05.933362 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:05.937457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d393f-16db-42a2-9cb8-ce6ee0a41b59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:05.939358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d393f-16db-42a2-9cb8-ce6ee0a41b59 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:05.957987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 674/1349] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:05:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:05:10.925186 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83b1eb-733f-494b-a1a5-4f6d83bf82eb None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:10.926026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83b1eb-733f-494b-a1a5-4f6d83bf82eb None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:10.926463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83b1eb-733f-494b-a1a5-4f6d83bf82eb None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:10.929155 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 676/1350] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jul 20 09:05:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:10.934469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d011a-9837-484a-822c-3ca7f0ba9a39 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:10.935356 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d011a-9837-484a-822c-3ca7f0ba9a39 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:10.935792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d011a-9837-484a-822c-3ca7f0ba9a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:11.061825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-fa1d011a-9837-484a-822c-3ca7f0ba9a39 None None] MFA Rules not processed for user `066f69a87f5f48c7b20c1e0fcf5c1107`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:11.063119 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-fa1d011a-9837-484a-822c-3ca7f0ba9a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:11.184948 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 675/1351] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jul 20 09:05:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:05:15.427601 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9aabf9-e551-4485-9c77-17940c24cd4f None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:15.428113 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9aabf9-e551-4485-9c77-17940c24cd4f None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:15.428113 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9aabf9-e551-4485-9c77-17940c24cd4f None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:15.429589 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 677/1352] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jul 20 09:05:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:15.434635 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc2a56-3f27-4c29-89f7-c14b84eb0ffa None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:15.435105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc2a56-3f27-4c29-89f7-c14b84eb0ffa None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:15.435464 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc2a56-3f27-4c29-89f7-c14b84eb0ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:15.491737 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:15.516504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-0acc2a56-3f27-4c29-89f7-c14b84eb0ffa None None] MFA Rules not processed for user `066f69a87f5f48c7b20c1e0fcf5c1107`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:15.518085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-0acc2a56-3f27-4c29-89f7-c14b84eb0ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:15.586729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=59 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:15.639757 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 676/1353] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jul 20 09:05:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:17.019456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cde05-e17d-4e38-ad91-ee882cd24e26 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:17.020057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cde05-e17d-4e38-ad91-ee882cd24e26 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:17.020057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cde05-e17d-4e38-ad91-ee882cd24e26 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:17.021636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 678/1354] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jul 20 09:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:17.026928 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c560e13-5520-415f-baf5-de3cb5e9d39e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:17.027378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c560e13-5520-415f-baf5-de3cb5e9d39e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:17.027614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c560e13-5520-415f-baf5-de3cb5e9d39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:17.103871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-5c560e13-5520-415f-baf5-de3cb5e9d39e None None] MFA Rules not processed for user `066f69a87f5f48c7b20c1e0fcf5c1107`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:17.105258 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5c560e13-5520-415f-baf5-de3cb5e9d39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:17.148092 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:17.214633 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 677/1355] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jul 20 09:05:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:20.645679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1752b-e585-4093-92fa-67197a314139 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:20.646615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1752b-e585-4093-92fa-67197a314139 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:20.647474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1752b-e585-4093-92fa-67197a314139 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:20.650743 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 679/1356] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:05:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:20.660619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d010be-6600-442b-a732-0c957d5baa46 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:20.661668 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d010be-6600-442b-a732-0c957d5baa46 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:20.664502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d010be-6600-442b-a732-0c957d5baa46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:20.737644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-98d010be-6600-442b-a732-0c957d5baa46 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:20.739254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-98d010be-6600-442b-a732-0c957d5baa46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:20.840962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 678/1357] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:20.899186 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b8191-2b74-45b5-a72d-98b6755acae2 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:20.899930 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b8191-2b74-45b5-a72d-98b6755acae2 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:20.900475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b8191-2b74-45b5-a72d-98b6755acae2 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:20.902549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 680/1358] 10.222.0.44 () {54 vars in 962 bytes} [Wed Jul 20 09:05:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:20.911297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd472d5-fb52-4d12-bde3-5192d8d9ff6b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:20.911589 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd472d5-fb52-4d12-bde3-5192d8d9ff6b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:20.912035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd472d5-fb52-4d12-bde3-5192d8d9ff6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:20.934591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:21.001193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-ebd472d5-fb52-4d12-bde3-5192d8d9ff6b None None] MFA Rules not processed for user `be08c43f0d0c4a8194596e54500b44ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:21.002411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ebd472d5-fb52-4d12-bde3-5192d8d9ff6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:21.128049 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 679/1359] 10.222.0.44 () {58 vars in 1059 bytes} [Wed Jul 20 09:05:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:21.137500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b8191-2b74-45b5-a72d-98b6755acae2 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:21.140681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c61b8191-2b74-45b5-a72d-98b6755acae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:21.296910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b8191-2b74-45b5-a72d-98b6755acae2 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:21.375953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:21.418573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b38f3-9e1a-4ac8-88e9-f4b96fc743f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:21.419258 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b38f3-9e1a-4ac8-88e9-f4b96fc743f3 service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:21.419540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b38f3-9e1a-4ac8-88e9-f4b96fc743f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:21.419830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b38f3-9e1a-4ac8-88e9-f4b96fc743f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:21.421440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-031b38f3-9e1a-4ac8-88e9-f4b96fc743f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:21.517383 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b38f3-9e1a-4ac8-88e9-f4b96fc743f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:21.519145 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b38f3-9e1a-4ac8-88e9-f4b96fc743f3 service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:21.544268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 681/1360] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:05:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:05:23.845998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51973b69-a8f9-4da2-b60a-22e68020fa28 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:23.846497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51973b69-a8f9-4da2-b60a-22e68020fa28 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:23.846497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51973b69-a8f9-4da2-b60a-22e68020fa28 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:23.848053 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 680/1361] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:05:23.853814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17398d-fb13-490a-8135-9e5fad0b830f None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:23.854227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17398d-fb13-490a-8135-9e5fad0b830f None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:23.854400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17398d-fb13-490a-8135-9e5fad0b830f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:23.923703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-dd17398d-fb13-490a-8135-9e5fad0b830f None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:23.924892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-dd17398d-fb13-490a-8135-9e5fad0b830f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:23.967047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:24.041923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 682/1362] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:05:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:24.089157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51973b69-a8f9-4da2-b60a-22e68020fa28 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:24.092200 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-51973b69-a8f9-4da2-b60a-22e68020fa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:24.166716 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:24.272928 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51973b69-a8f9-4da2-b60a-22e68020fa28 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:24.400792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b649b6b-34f9-4fec-aa93-d112eddd8b70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:24.401446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b649b6b-34f9-4fec-aa93-d112eddd8b70 service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:24.401727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b649b6b-34f9-4fec-aa93-d112eddd8b70 service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:24.402000 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b649b6b-34f9-4fec-aa93-d112eddd8b70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:24.403657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5b649b6b-34f9-4fec-aa93-d112eddd8b70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:24.427402 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:24.497223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b649b6b-34f9-4fec-aa93-d112eddd8b70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:24.498845 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b649b6b-34f9-4fec-aa93-d112eddd8b70 service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:24.517820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 681/1363] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:05:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:05:27.026210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38707245-79ff-443a-ad45-682dfec26756 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:27.026771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38707245-79ff-443a-ad45-682dfec26756 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:27.027212 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38707245-79ff-443a-ad45-682dfec26756 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:27.029702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 683/1364] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:05:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:27.035373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9c519-9a7b-4031-983d-c57a07cf2f1a None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:27.036400 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9c519-9a7b-4031-983d-c57a07cf2f1a None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:27.037104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9c519-9a7b-4031-983d-c57a07cf2f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:27.107708 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-00a9c519-9a7b-4031-983d-c57a07cf2f1a None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:27.108911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-00a9c519-9a7b-4031-983d-c57a07cf2f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:27.213986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:27.221502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 682/1365] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:05:27.264259 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f3e2a-1a5b-4e0b-ab9e-771c7dd1bb5d None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:27.264640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f3e2a-1a5b-4e0b-ab9e-771c7dd1bb5d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:27.264992 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f3e2a-1a5b-4e0b-ab9e-771c7dd1bb5d None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:27.266784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 684/1366] 10.222.0.44 () {54 vars in 962 bytes} [Wed Jul 20 09:05:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:27.272927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82d82d-f1fa-4119-83e0-9425d9f00ba7 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:27.273224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82d82d-f1fa-4119-83e0-9425d9f00ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:27.273507 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82d82d-f1fa-4119-83e0-9425d9f00ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:27.363180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-4f82d82d-f1fa-4119-83e0-9425d9f00ba7 None None] MFA Rules not processed for user `be08c43f0d0c4a8194596e54500b44ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:27.364460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-4f82d82d-f1fa-4119-83e0-9425d9f00ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:27.498627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 683/1367] 10.222.0.44 () {58 vars in 1059 bytes} [Wed Jul 20 09:05:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:27.507476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f3e2a-1a5b-4e0b-ab9e-771c7dd1bb5d None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:27.509104 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-9a7f3e2a-1a5b-4e0b-ab9e-771c7dd1bb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:27.680157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f3e2a-1a5b-4e0b-ab9e-771c7dd1bb5d None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:27.702440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:27.802304 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b902d3-4099-4d0b-b257-d0093ca31dae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:27.803129 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b902d3-4099-4d0b-b257-d0093ca31dae service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:27.803413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b902d3-4099-4d0b-b257-d0093ca31dae service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:27.803692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b902d3-4099-4d0b-b257-d0093ca31dae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:27.805340 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-88b902d3-4099-4d0b-b257-d0093ca31dae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:27.900303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b902d3-4099-4d0b-b257-d0093ca31dae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:27.901953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b902d3-4099-4d0b-b257-d0093ca31dae service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:27.921943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 685/1368] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:05:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:05:28.821728 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbdab2-e38f-4020-b21a-bde9788f1582 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:28.822243 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbdab2-e38f-4020-b21a-bde9788f1582 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:28.822243 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fbdab2-e38f-4020-b21a-bde9788f1582 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:28.823794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 684/1369] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:05:28.828841 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b1c3c-0da4-47de-bddb-5eb71a15cd23 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:28.829232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b1c3c-0da4-47de-bddb-5eb71a15cd23 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:28.829400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b1c3c-0da4-47de-bddb-5eb71a15cd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:28.860822 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:28.898054 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-c16b1c3c-0da4-47de-bddb-5eb71a15cd23 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:28.899297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c16b1c3c-0da4-47de-bddb-5eb71a15cd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:29.003202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 686/1370] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:05:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:29.347307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb718a9-dca8-4bb9-ab24-4cc4c77dc7b9 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:29.347307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb718a9-dca8-4bb9-ab24-4cc4c77dc7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:29.347307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb718a9-dca8-4bb9-ab24-4cc4c77dc7b9 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:29.351040 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 685/1371] 10.222.0.44 () {54 vars in 964 bytes} [Wed Jul 20 09:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:29.353757 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e51bb-db04-4cff-883f-c5fa1616a528 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:29.354048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e51bb-db04-4cff-883f-c5fa1616a528 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:29.354324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e51bb-db04-4cff-883f-c5fa1616a528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:29.433370 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-4e7e51bb-db04-4cff-883f-c5fa1616a528 None None] MFA Rules not processed for user `99dc3db93e9045b8b3ef59a030822972`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:29.434565 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4e7e51bb-db04-4cff-883f-c5fa1616a528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:29.484819 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:29.516645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 687/1372] 10.222.0.44 () {58 vars in 1061 bytes} [Wed Jul 20 09:05:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:29.524380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb718a9-dca8-4bb9-ab24-4cc4c77dc7b9 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:29.525872 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5fb718a9-dca8-4bb9-ab24-4cc4c77dc7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:29.583866 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:29.641900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb718a9-dca8-4bb9-ab24-4cc4c77dc7b9 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:29.721003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca112b42-b8d6-4cca-821a-deaa4d836643 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:29.721659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca112b42-b8d6-4cca-821a-deaa4d836643 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:29.721947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca112b42-b8d6-4cca-821a-deaa4d836643 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:29.722229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca112b42-b8d6-4cca-821a-deaa4d836643 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:29.724592 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ca112b42-b8d6-4cca-821a-deaa4d836643 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:29.835796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:29.852322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca112b42-b8d6-4cca-821a-deaa4d836643 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:29.853799 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca112b42-b8d6-4cca-821a-deaa4d836643 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:29.873778 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 686/1373] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:05:30.050669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09524872-d61b-4f19-b13c-4f7bb6422313 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:30.050669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09524872-d61b-4f19-b13c-4f7bb6422313 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:30.050669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09524872-d61b-4f19-b13c-4f7bb6422313 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:30.050669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 688/1374] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:30.057160 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189465e-be38-4b2e-88be-3df3d9d3895c None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:30.057437 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189465e-be38-4b2e-88be-3df3d9d3895c None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:30.057755 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189465e-be38-4b2e-88be-3df3d9d3895c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:30.123962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-c189465e-be38-4b2e-88be-3df3d9d3895c None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:30.125340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c189465e-be38-4b2e-88be-3df3d9d3895c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:30.238488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 687/1375] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:30.282829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09524872-d61b-4f19-b13c-4f7bb6422313 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:30.284336 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-09524872-d61b-4f19-b13c-4f7bb6422313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:30.429463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09524872-d61b-4f19-b13c-4f7bb6422313 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:30.480753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:30.545547 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a41c9-6b7e-4db7-b520-ebaefb0a8535 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:30.546197 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a41c9-6b7e-4db7-b520-ebaefb0a8535 service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:30.546482 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a41c9-6b7e-4db7-b520-ebaefb0a8535 service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:30.546789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a41c9-6b7e-4db7-b520-ebaefb0a8535 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:30.548519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f03a41c9-6b7e-4db7-b520-ebaefb0a8535 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:30.644256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a41c9-6b7e-4db7-b520-ebaefb0a8535 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:30.646036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a41c9-6b7e-4db7-b520-ebaefb0a8535 service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:30.664172 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 689/1376] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:05:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:05:32.741077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5ffcc-e88d-4042-bacd-8a094c79d828 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:32.741571 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5ffcc-e88d-4042-bacd-8a094c79d828 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:32.741688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5ffcc-e88d-4042-bacd-8a094c79d828 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:32.743283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 688/1377] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:32.748614 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134d127-9df5-4765-bc72-56e25723ef58 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:32.749026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134d127-9df5-4765-bc72-56e25723ef58 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:32.749186 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134d127-9df5-4765-bc72-56e25723ef58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:32.814589 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-6134d127-9df5-4765-bc72-56e25723ef58 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:32.815781 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6134d127-9df5-4765-bc72-56e25723ef58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:32.822756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:32.924630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 690/1378] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:32.969361 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5ffcc-e88d-4042-bacd-8a094c79d828 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:32.970904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-3ce5ffcc-e88d-4042-bacd-8a094c79d828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:33.032260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:33.119949 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5ffcc-e88d-4042-bacd-8a094c79d828 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:33.241752 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fb6b5-f5ce-4ee9-8fd5-391071013cc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:33.242410 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fb6b5-f5ce-4ee9-8fd5-391071013cc4 service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:33.242725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fb6b5-f5ce-4ee9-8fd5-391071013cc4 service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:33.243011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fb6b5-f5ce-4ee9-8fd5-391071013cc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:33.244665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-db0fb6b5-f5ce-4ee9-8fd5-391071013cc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:33.266420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:33.372714 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0fb6b5-f5ce-4ee9-8fd5-391071013cc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:33.374219 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0fb6b5-f5ce-4ee9-8fd5-391071013cc4 service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:33.397328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 689/1379] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:05:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:05:35.511964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e93fb9-17f4-4bfb-a0e3-14f16913425b None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:35.512526 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e93fb9-17f4-4bfb-a0e3-14f16913425b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:35.512804 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e93fb9-17f4-4bfb-a0e3-14f16913425b None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:35.514561 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 691/1380] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:05:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:35.520868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75b6a6-71af-40d9-9ea8-d38585a18de7 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:35.521448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75b6a6-71af-40d9-9ea8-d38585a18de7 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:35.521448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75b6a6-71af-40d9-9ea8-d38585a18de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:35.598300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-6d75b6a6-71af-40d9-9ea8-d38585a18de7 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:35.599745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6d75b6a6-71af-40d9-9ea8-d38585a18de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:35.696665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:35.707085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 690/1381] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:35.748376 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e93fb9-17f4-4bfb-a0e3-14f16913425b None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:35.749878 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b3e93fb9-17f4-4bfb-a0e3-14f16913425b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:35.903005 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:35.914974 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e93fb9-17f4-4bfb-a0e3-14f16913425b None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:36.039560 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc274f-292b-494b-8023-c90aa41daa57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:36.040218 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc274f-292b-494b-8023-c90aa41daa57 service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:36.040503 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc274f-292b-494b-8023-c90aa41daa57 service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:36.040778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc274f-292b-494b-8023-c90aa41daa57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:36.042409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-7cdc274f-292b-494b-8023-c90aa41daa57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:36.138945 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc274f-292b-494b-8023-c90aa41daa57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:36.140660 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc274f-292b-494b-8023-c90aa41daa57 service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:36.151875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:36.162996 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 692/1382] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:05:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:05:38.281026 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652652b7-5904-43af-b9ec-7d3186374d3c None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:38.281759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652652b7-5904-43af-b9ec-7d3186374d3c None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:38.281759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652652b7-5904-43af-b9ec-7d3186374d3c None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:38.283572 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 691/1383] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:38.289520 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09539d2-c853-409b-9547-dbe76a332826 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:38.290026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09539d2-c853-409b-9547-dbe76a332826 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:38.290112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09539d2-c853-409b-9547-dbe76a332826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:38.361362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a09539d2-c853-409b-9547-dbe76a332826 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:38.362575 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a09539d2-c853-409b-9547-dbe76a332826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:38.463012 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 693/1384] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:05:38.505982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652652b7-5904-43af-b9ec-7d3186374d3c None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:38.507521 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-652652b7-5904-43af-b9ec-7d3186374d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:38.660928 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652652b7-5904-43af-b9ec-7d3186374d3c None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:38.731721 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:38.780331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a61faf-f16a-49df-a69f-bea076baa803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:38.780976 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a61faf-f16a-49df-a69f-bea076baa803 service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:38.781260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a61faf-f16a-49df-a69f-bea076baa803 service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:38.781545 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a61faf-f16a-49df-a69f-bea076baa803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:38.783212 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f3a61faf-f16a-49df-a69f-bea076baa803 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:38.877877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a61faf-f16a-49df-a69f-bea076baa803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:38.879879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a61faf-f16a-49df-a69f-bea076baa803 service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:38.899008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 692/1385] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:05:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:05:41.309134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49991f-2654-4e47-8190-3320905d164a None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:41.309754 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49991f-2654-4e47-8190-3320905d164a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:41.310152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49991f-2654-4e47-8190-3320905d164a None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:41.312456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 694/1386] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:05:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:41.321325 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b4461-070e-4f60-98f6-ad5398cbce93 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:41.321835 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b4461-070e-4f60-98f6-ad5398cbce93 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:41.321920 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b4461-070e-4f60-98f6-ad5398cbce93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:41.393552 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-e39b4461-070e-4f60-98f6-ad5398cbce93 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:41.395355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e39b4461-070e-4f60-98f6-ad5398cbce93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:41.430155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:41.501114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 693/1387] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:41.546896 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49991f-2654-4e47-8190-3320905d164a None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:41.549128 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-7b49991f-2654-4e47-8190-3320905d164a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:41.639383 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:41.714668 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49991f-2654-4e47-8190-3320905d164a None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:41.830591 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298459ed-6a8c-40f0-9de1-9418986a6bec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:41.831281 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298459ed-6a8c-40f0-9de1-9418986a6bec service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:41.831563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298459ed-6a8c-40f0-9de1-9418986a6bec service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:41.831841 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298459ed-6a8c-40f0-9de1-9418986a6bec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:41.834142 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-298459ed-6a8c-40f0-9de1-9418986a6bec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:41.872926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:41.932679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298459ed-6a8c-40f0-9de1-9418986a6bec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:41.935006 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298459ed-6a8c-40f0-9de1-9418986a6bec service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:41.959850 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 695/1388] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:05:44.321111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7fee1-89ee-4965-bd08-96f4b7dd60c3 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:44.321644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7fee1-89ee-4965-bd08-96f4b7dd60c3 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:44.321739 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7fee1-89ee-4965-bd08-96f4b7dd60c3 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:44.323297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 694/1389] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:05:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:44.328553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bac38a-4b50-4516-990f-3eeb1f1c87f6 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:44.329309 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bac38a-4b50-4516-990f-3eeb1f1c87f6 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:44.329593 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bac38a-4b50-4516-990f-3eeb1f1c87f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:44.394287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-09bac38a-4b50-4516-990f-3eeb1f1c87f6 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:44.396271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-09bac38a-4b50-4516-990f-3eeb1f1c87f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:44.491670 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 696/1390] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:44.539837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7fee1-89ee-4965-bd08-96f4b7dd60c3 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:44.541459 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e8f7fee1-89ee-4965-bd08-96f4b7dd60c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:44.687238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7fee1-89ee-4965-bd08-96f4b7dd60c3 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:44.704931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:44.805278 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed43a53-76e0-4ff4-ae32-62a0250361b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:44.805925 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed43a53-76e0-4ff4-ae32-62a0250361b9 service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:44.806205 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed43a53-76e0-4ff4-ae32-62a0250361b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:44.806483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed43a53-76e0-4ff4-ae32-62a0250361b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:44.808137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2ed43a53-76e0-4ff4-ae32-62a0250361b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:44.897779 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed43a53-76e0-4ff4-ae32-62a0250361b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:44.899570 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed43a53-76e0-4ff4-ae32-62a0250361b9 service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:44.918068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 695/1391] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:05:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:05:45.140174 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e61fa-39f4-4f2c-a56f-d346d1b9078f None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:45.140478 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e61fa-39f4-4f2c-a56f-d346d1b9078f None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:45.140762 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e61fa-39f4-4f2c-a56f-d346d1b9078f None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:45.142155 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 697/1392] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:45.148274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b7272-77ac-4183-a6c0-e324dcd9ede6 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:45.148572 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b7272-77ac-4183-a6c0-e324dcd9ede6 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:45.148856 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b7272-77ac-4183-a6c0-e324dcd9ede6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:45.173852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:45.217907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-5e8b7272-77ac-4183-a6c0-e324dcd9ede6 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:45.219114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5e8b7272-77ac-4183-a6c0-e324dcd9ede6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:45.318865 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 696/1393] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:45.375777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411daec-9eec-4115-91df-bb8f623689f4 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:45.376247 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411daec-9eec-4115-91df-bb8f623689f4 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:45.376340 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411daec-9eec-4115-91df-bb8f623689f4 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:45.377684 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 698/1394] 10.222.0.44 () {54 vars in 966 bytes} [Wed Jul 20 09:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:45.383675 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0411d-acbf-4a98-b64a-bb563b53e19e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:45.384109 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0411d-acbf-4a98-b64a-bb563b53e19e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:45.384230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0411d-acbf-4a98-b64a-bb563b53e19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:45.461098 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-46d0411d-acbf-4a98-b64a-bb563b53e19e None None] MFA Rules not processed for user `bb93eb1d25c4410591125b0ab7a73149`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:45.462296 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-46d0411d-acbf-4a98-b64a-bb563b53e19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:45.502403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:45.540609 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 697/1395] 10.222.0.44 () {58 vars in 1063 bytes} [Wed Jul 20 09:05:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:45.547896 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411daec-9eec-4115-91df-bb8f623689f4 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:45.549406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4411daec-9eec-4115-91df-bb8f623689f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:45.563502 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:45.654883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411daec-9eec-4115-91df-bb8f623689f4 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:45.731331 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ef5eb-17be-410e-8501-1e3e1f192928 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb93eb1d25c4410591125b0ab7a73149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:45.731985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ef5eb-17be-410e-8501-1e3e1f192928 service glance] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:45.732271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ef5eb-17be-410e-8501-1e3e1f192928 service glance] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:45.732554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ef5eb-17be-410e-8501-1e3e1f192928 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:45.734194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-fb9ef5eb-17be-410e-8501-1e3e1f192928 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:45.802348 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:45.843546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ef5eb-17be-410e-8501-1e3e1f192928 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:45.844996 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ef5eb-17be-410e-8501-1e3e1f192928 service glance] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:45.863021 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 699/1396] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:05:46.583414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9055be2-90f4-4ed6-9a49-b07a28726c7e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:46.583871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9055be2-90f4-4ed6-9a49-b07a28726c7e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:46.583960 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9055be2-90f4-4ed6-9a49-b07a28726c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:46.656541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-d9055be2-90f4-4ed6-9a49-b07a28726c7e None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:46.657754 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d9055be2-90f4-4ed6-9a49-b07a28726c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:46.747131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 698/1397] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jul 20 09:05:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:46.793530 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14fcd8-a8ae-4266-bfcb-10612d4bfd7c None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:46.794055 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14fcd8-a8ae-4266-bfcb-10612d4bfd7c None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:46.794242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14fcd8-a8ae-4266-bfcb-10612d4bfd7c None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:46.795709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 700/1398] 10.222.0.44 () {54 vars in 943 bytes} [Wed Jul 20 09:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:05:46.802176 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a7c3a-19c9-4ca3-bc80-0d457dd966bb None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:46.802479 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a7c3a-19c9-4ca3-bc80-0d457dd966bb None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:46.802856 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a7c3a-19c9-4ca3-bc80-0d457dd966bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:46.870300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:46.878283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-771a7c3a-19c9-4ca3-bc80-0d457dd966bb None None] MFA Rules not processed for user `03b4375e6ead42fb8d77a7b30617cb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:46.880090 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-771a7c3a-19c9-4ca3-bc80-0d457dd966bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:46.988052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 699/1399] 10.222.0.44 () {58 vars in 1040 bytes} [Wed Jul 20 09:05:46 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:46.997452 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14fcd8-a8ae-4266-bfcb-10612d4bfd7c None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:46.998989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ed14fcd8-a8ae-4266-bfcb-10612d4bfd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:47.003680 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e799552-650f-472c-bf95-4c4746b23502 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:47.004582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e799552-650f-472c-bf95-4c4746b23502 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:47.005089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e799552-650f-472c-bf95-4c4746b23502 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:47.006597 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 700/1400] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:05:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:05:47.011536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93381ee0-10bd-4061-aa90-8ed571bac24b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:47.011832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93381ee0-10bd-4061-aa90-8ed571bac24b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:47.012123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93381ee0-10bd-4061-aa90-8ed571bac24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:47.078787 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-93381ee0-10bd-4061-aa90-8ed571bac24b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:47.079918 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-93381ee0-10bd-4061-aa90-8ed571bac24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:47.136146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14fcd8-a8ae-4266-bfcb-10612d4bfd7c None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:47.142044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:47.172489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:47.178803 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 701/1401] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:05:47.223630 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93381ee0-10bd-4061-aa90-8ed571bac24b None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:47.224829 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-93381ee0-10bd-4061-aa90-8ed571bac24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:47.248406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd18f2-5d4e-458f-a6b5-29bbf9f4fbce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:47.249069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd18f2-5d4e-458f-a6b5-29bbf9f4fbce service swift] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:47.249385 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd18f2-5d4e-458f-a6b5-29bbf9f4fbce service swift] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:47.249655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd18f2-5d4e-458f-a6b5-29bbf9f4fbce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:47.251371 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-bffd18f2-5d4e-458f-a6b5-29bbf9f4fbce service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:47.336595 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd18f2-5d4e-458f-a6b5-29bbf9f4fbce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:47.338142 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd18f2-5d4e-458f-a6b5-29bbf9f4fbce service swift] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:47.343223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93381ee0-10bd-4061-aa90-8ed571bac24b None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:47.346287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 701/1402] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:05:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:05:47.423946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:47.470669 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a6be2-638f-4c06-92be-96112d86f80a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:47.471483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a6be2-638f-4c06-92be-96112d86f80a service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:47.471829 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a6be2-638f-4c06-92be-96112d86f80a service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:47.472185 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a6be2-638f-4c06-92be-96112d86f80a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:47.475183 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9b2a6be2-638f-4c06-92be-96112d86f80a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:47.584521 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a6be2-638f-4c06-92be-96112d86f80a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:47.586119 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a6be2-638f-4c06-92be-96112d86f80a service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:47.610769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 702/1403] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:05:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:05:49.745847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178ce88-6020-4167-bf4b-0a7f2a47f556 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:49.746448 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178ce88-6020-4167-bf4b-0a7f2a47f556 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:49.746448 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178ce88-6020-4167-bf4b-0a7f2a47f556 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:49.748228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 702/1404] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:49.753207 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a924dfa-38ec-48f7-9c3c-f1a46fc4b219 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:49.753621 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a924dfa-38ec-48f7-9c3c-f1a46fc4b219 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:49.754451 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a924dfa-38ec-48f7-9c3c-f1a46fc4b219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:49.823418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-9a924dfa-38ec-48f7-9c3c-f1a46fc4b219 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:49.824549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9a924dfa-38ec-48f7-9c3c-f1a46fc4b219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:49.866742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:49.930592 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 703/1405] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:49.975456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178ce88-6020-4167-bf4b-0a7f2a47f556 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:49.976997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c178ce88-6020-4167-bf4b-0a7f2a47f556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:50.056577 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:50.140720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178ce88-6020-4167-bf4b-0a7f2a47f556 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:50.291032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda3d87-4ff0-42af-8deb-09e31b21ff31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:50.291691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda3d87-4ff0-42af-8deb-09e31b21ff31 service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:50.291995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda3d87-4ff0-42af-8deb-09e31b21ff31 service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:50.292272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda3d87-4ff0-42af-8deb-09e31b21ff31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:50.293905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-9fda3d87-4ff0-42af-8deb-09e31b21ff31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:50.331643 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:50.409456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda3d87-4ff0-42af-8deb-09e31b21ff31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:50.411232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda3d87-4ff0-42af-8deb-09e31b21ff31 service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:50.436068 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 703/1406] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:05:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:05:52.526334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd131a76-0969-4e6e-a9fb-4adf34f6a113 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:52.526944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd131a76-0969-4e6e-a9fb-4adf34f6a113 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:52.526944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd131a76-0969-4e6e-a9fb-4adf34f6a113 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:52.528938 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 704/1407] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:05:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:05:52.534579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250e531-36f8-4dba-8ddf-b6cab8c8e042 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:52.535764 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250e531-36f8-4dba-8ddf-b6cab8c8e042 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:52.536048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250e531-36f8-4dba-8ddf-b6cab8c8e042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:52.606294 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-3250e531-36f8-4dba-8ddf-b6cab8c8e042 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:05:52.607550 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3250e531-36f8-4dba-8ddf-b6cab8c8e042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:52.730367 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 704/1408] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:05:52.776794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd131a76-0969-4e6e-a9fb-4adf34f6a113 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:05:52.779906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-cd131a76-0969-4e6e-a9fb-4adf34f6a113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:52.941604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd131a76-0969-4e6e-a9fb-4adf34f6a113 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:05:52.966814 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:05:53.071903 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f282a5-4c4c-4c8a-be14-a071ef22d947 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:05:53.072842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f282a5-4c4c-4c8a-be14-a071ef22d947 service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:05:53.073124 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f282a5-4c4c-4c8a-be14-a071ef22d947 service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:05:53.073400 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f282a5-4c4c-4c8a-be14-a071ef22d947 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:05:53.075068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-46f282a5-4c4c-4c8a-be14-a071ef22d947 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:05:53.187728 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f282a5-4c4c-4c8a-be14-a071ef22d947 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:05:53.189984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f282a5-4c4c-4c8a-be14-a071ef22d947 service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:05:53.214831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 705/1409] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:05:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:02.727917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:03.207312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=88 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:09.752391 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a1d6c-069f-4f23-8197-277de7e3c0b6 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:09.752969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a1d6c-069f-4f23-8197-277de7e3c0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:09.752969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a1d6c-069f-4f23-8197-277de7e3c0b6 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:09.754370 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 705/1410] 10.222.0.44 () {54 vars in 918 bytes} [Wed Jul 20 09:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:09.759632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4decd0c-61be-4af0-96a1-09e8bd7e3095 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:09.760088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4decd0c-61be-4af0-96a1-09e8bd7e3095 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:09.760186 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4decd0c-61be-4af0-96a1-09e8bd7e3095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:09.834136 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-b4decd0c-61be-4af0-96a1-09e8bd7e3095 None None] MFA Rules not processed for user `066f69a87f5f48c7b20c1e0fcf5c1107`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:09.835354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b4decd0c-61be-4af0-96a1-09e8bd7e3095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:09.940607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 706/1411] 10.222.0.44 () {58 vars in 1015 bytes} [Wed Jul 20 09:06:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:06:09.954957 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9650491-7c2c-420d-abc2-2ab04f55a807 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:09.955253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9650491-7c2c-420d-abc2-2ab04f55a807 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:09.955540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9650491-7c2c-420d-abc2-2ab04f55a807 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:09.957421 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 706/1412] 10.222.0.44 () {54 vars in 918 bytes} [Wed Jul 20 09:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:09.962115 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0944ea-4927-4697-9517-6216a8c88e53 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:09.962442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0944ea-4927-4697-9517-6216a8c88e53 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:09.962752 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0944ea-4927-4697-9517-6216a8c88e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:10.034456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:10.039134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-5c0944ea-4927-4697-9517-6216a8c88e53 None None] MFA Rules not processed for user `be08c43f0d0c4a8194596e54500b44ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:10.040919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5c0944ea-4927-4697-9517-6216a8c88e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:10.157103 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 707/1413] 10.222.0.44 () {58 vars in 1015 bytes} [Wed Jul 20 09:06:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:06:12.653610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa5e09-6e9e-499e-af33-f91c8d7b839a None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:12.653610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa5e09-6e9e-499e-af33-f91c8d7b839a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:12.654242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa5e09-6e9e-499e-af33-f91c8d7b839a None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:12.655493 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:12.661231 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35912b-907e-402b-98e5-b3bab66c4a09 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:12.661825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35912b-907e-402b-98e5-b3bab66c4a09 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:12.661825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35912b-907e-402b-98e5-b3bab66c4a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:12.726613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-0d35912b-907e-402b-98e5-b3bab66c4a09 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:12.727835 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-0d35912b-907e-402b-98e5-b3bab66c4a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:12.783495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:12.825866 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:06:12.869111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa5e09-6e9e-499e-af33-f91c8d7b839a None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:12.870830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-2bfa5e09-6e9e-499e-af33-f91c8d7b839a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:13.018043 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa5e09-6e9e-499e-af33-f91c8d7b839a None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:13.090327 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:13.130061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea05e24-94a4-4d0e-9904-8217acabf45e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:13.130838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea05e24-94a4-4d0e-9904-8217acabf45e service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:13.131128 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea05e24-94a4-4d0e-9904-8217acabf45e service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:13.131408 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea05e24-94a4-4d0e-9904-8217acabf45e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:13.133045 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4ea05e24-94a4-4d0e-9904-8217acabf45e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:13.220230 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea05e24-94a4-4d0e-9904-8217acabf45e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:13.222389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea05e24-94a4-4d0e-9904-8217acabf45e service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:13.240654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 708/1416] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:14.146414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eab9d8-5406-454f-ab26-cfc42cb18304 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:14.147249 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eab9d8-5406-454f-ab26-cfc42cb18304 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:14.147249 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eab9d8-5406-454f-ab26-cfc42cb18304 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:14.148442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:14.153500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae11a3-41d9-409e-a251-5220867a6da4 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:14.153905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae11a3-41d9-409e-a251-5220867a6da4 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:14.154070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae11a3-41d9-409e-a251-5220867a6da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:14.227523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-b3ae11a3-41d9-409e-a251-5220867a6da4 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:14.228750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b3ae11a3-41d9-409e-a251-5220867a6da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:14.258321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:14.337503 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:06:14.502417 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eab9d8-5406-454f-ab26-cfc42cb18304 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:14.504299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-01eab9d8-5406-454f-ab26-cfc42cb18304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:14.657298 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eab9d8-5406-454f-ab26-cfc42cb18304 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:14.700433 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:14.778835 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b7886-8a1d-45da-856a-a19908f2c8d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:14.779742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b7886-8a1d-45da-856a-a19908f2c8d1 service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:14.780257 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b7886-8a1d-45da-856a-a19908f2c8d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:14.780768 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b7886-8a1d-45da-856a-a19908f2c8d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:14.783965 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d33b7886-8a1d-45da-856a-a19908f2c8d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:14.890218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b7886-8a1d-45da-856a-a19908f2c8d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:14.891892 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b7886-8a1d-45da-856a-a19908f2c8d1 service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:14.912627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 710/1419] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:15.816764 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47973f5-ba3e-4941-9031-710c87e36eca None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:15.817284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47973f5-ba3e-4941-9031-710c87e36eca None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:15.817284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47973f5-ba3e-4941-9031-710c87e36eca None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:15.818619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:15.823887 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f6725-cbaa-4f40-bc10-8071b7ff7a5b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:15.824288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f6725-cbaa-4f40-bc10-8071b7ff7a5b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:15.824454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f6725-cbaa-4f40-bc10-8071b7ff7a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:15.877572 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:15.899141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-3d5f6725-cbaa-4f40-bc10-8071b7ff7a5b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:15.900356 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-3d5f6725-cbaa-4f40-bc10-8071b7ff7a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:15.993968 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:06:16.149529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47973f5-ba3e-4941-9031-710c87e36eca None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:16.151032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f47973f5-ba3e-4941-9031-710c87e36eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:16.294772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47973f5-ba3e-4941-9031-710c87e36eca None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:16.306697 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:16.407181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72799d-25c0-4890-a351-ed678050cd60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:16.407747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72799d-25c0-4890-a351-ed678050cd60 service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:16.408029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72799d-25c0-4890-a351-ed678050cd60 service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:16.408303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72799d-25c0-4890-a351-ed678050cd60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:16.410025 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4a72799d-25c0-4890-a351-ed678050cd60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:16.497335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72799d-25c0-4890-a351-ed678050cd60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:16.498944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72799d-25c0-4890-a351-ed678050cd60 service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:16.516701 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 711/1422] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:17.381418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffd4c5-78b5-4d49-b781-d95ba48c2735 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:17.381969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffd4c5-78b5-4d49-b781-d95ba48c2735 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:17.381969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffd4c5-78b5-4d49-b781-d95ba48c2735 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:17.383437 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:17.388286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3607263-5e69-44a9-a698-cf2fb5fd7a9e None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:17.388738 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3607263-5e69-44a9-a698-cf2fb5fd7a9e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:17.388839 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3607263-5e69-44a9-a698-cf2fb5fd7a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:17.408642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:17.464706 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a3607263-5e69-44a9-a698-cf2fb5fd7a9e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:17.465905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a3607263-5e69-44a9-a698-cf2fb5fd7a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:17.567577 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:06:17.718050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffd4c5-78b5-4d49-b781-d95ba48c2735 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:17.719512 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-63ffd4c5-78b5-4d49-b781-d95ba48c2735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:17.841080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:17.878517 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffd4c5-78b5-4d49-b781-d95ba48c2735 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:17.993041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761af3c-04f5-48fc-a322-f7cd255809ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:17.993618 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761af3c-04f5-48fc-a322-f7cd255809ce service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:17.993902 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761af3c-04f5-48fc-a322-f7cd255809ce service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:17.994182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761af3c-04f5-48fc-a322-f7cd255809ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:17.995917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d761af3c-04f5-48fc-a322-f7cd255809ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:18.073088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:18.087402 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761af3c-04f5-48fc-a322-f7cd255809ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:18.089014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761af3c-04f5-48fc-a322-f7cd255809ce service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:18.107027 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 713/1425] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:18.980676 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2023dc-8bfe-42d6-a12b-a95fe2c0867a None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:18.981238 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2023dc-8bfe-42d6-a12b-a95fe2c0867a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:18.981238 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2023dc-8bfe-42d6-a12b-a95fe2c0867a None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:18.982611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:18.987979 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f67d83-8fbf-4c01-831a-c25d2241dce7 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:18.988411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f67d83-8fbf-4c01-831a-c25d2241dce7 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:18.988540 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f67d83-8fbf-4c01-831a-c25d2241dce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:19.062063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-82f67d83-8fbf-4c01-831a-c25d2241dce7 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:19.063304 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-82f67d83-8fbf-4c01-831a-c25d2241dce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:19.160671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:06:19.314055 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2023dc-8bfe-42d6-a12b-a95fe2c0867a None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:19.316390 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-1d2023dc-8bfe-42d6-a12b-a95fe2c0867a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:19.398889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:19.468759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2023dc-8bfe-42d6-a12b-a95fe2c0867a None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:19.583190 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec07436-049f-4719-9e30-a14f4869ccaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:19.583779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec07436-049f-4719-9e30-a14f4869ccaa service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:19.584059 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec07436-049f-4719-9e30-a14f4869ccaa service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:19.584333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec07436-049f-4719-9e30-a14f4869ccaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:19.585935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0ec07436-049f-4719-9e30-a14f4869ccaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:19.632772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:19.674197 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec07436-049f-4719-9e30-a14f4869ccaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:19.675985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec07436-049f-4719-9e30-a14f4869ccaa service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:19.693521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 714/1428] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:20.495250 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d8423-a861-4210-ad27-b10b664e097f None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:20.495906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d8423-a861-4210-ad27-b10b664e097f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:20.495906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d8423-a861-4210-ad27-b10b664e097f None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:20.497217 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:20.502138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef91e2-f418-4396-b82f-d71091e76d10 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:20.502540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef91e2-f418-4396-b82f-d71091e76d10 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:20.502757 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef91e2-f418-4396-b82f-d71091e76d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:20.576604 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-09ef91e2-f418-4396-b82f-d71091e76d10 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:20.577811 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-09ef91e2-f418-4396-b82f-d71091e76d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:20.673174 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:06:20.823518 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d8423-a861-4210-ad27-b10b664e097f None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:20.824940 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-3c7d8423-a861-4210-ad27-b10b664e097f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:20.874025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:20.970467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d8423-a861-4210-ad27-b10b664e097f None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:21.082915 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b933df-eecb-44d0-b818-aa80bb6fb530 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:21.083490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b933df-eecb-44d0-b818-aa80bb6fb530 service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:21.083772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b933df-eecb-44d0-b818-aa80bb6fb530 service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:21.084047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b933df-eecb-44d0-b818-aa80bb6fb530 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:21.085749 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-58b933df-eecb-44d0-b818-aa80bb6fb530 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:21.092864 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:21.175846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b933df-eecb-44d0-b818-aa80bb6fb530 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:21.177550 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b933df-eecb-44d0-b818-aa80bb6fb530 service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:21.196779 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 716/1431] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:22.004928 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983922a-be63-4211-8d1e-a43b408426a2 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:22.005506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983922a-be63-4211-8d1e-a43b408426a2 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:22.005506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983922a-be63-4211-8d1e-a43b408426a2 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:22.006886 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:06:22.011874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2971776-5090-4418-95f4-77a4da80add2 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:22.012282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2971776-5090-4418-95f4-77a4da80add2 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:22.012452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2971776-5090-4418-95f4-77a4da80add2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:22.087593 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-a2971776-5090-4418-95f4-77a4da80add2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:22.088803 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-a2971776-5090-4418-95f4-77a4da80add2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:22.156052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:22.185870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:06:22.339737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983922a-be63-4211-8d1e-a43b408426a2 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:22.341156 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f983922a-be63-4211-8d1e-a43b408426a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:22.358837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:22.486199 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983922a-be63-4211-8d1e-a43b408426a2 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:22.572678 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:22.598285 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1175c0-e178-40b7-a75a-f3e4b18792b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:22.598967 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1175c0-e178-40b7-a75a-f3e4b18792b6 service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:22.599248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1175c0-e178-40b7-a75a-f3e4b18792b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:22.599521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1175c0-e178-40b7-a75a-f3e4b18792b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:22.601136 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-cf1175c0-e178-40b7-a75a-f3e4b18792b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:22.692024 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1175c0-e178-40b7-a75a-f3e4b18792b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:22.693623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1175c0-e178-40b7-a75a-f3e4b18792b6 service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:22.711809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 717/1434] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:23.523665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7849608-48a3-4c59-b484-19a38fb7896f None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:23.524197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7849608-48a3-4c59-b484-19a38fb7896f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:23.524197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7849608-48a3-4c59-b484-19a38fb7896f None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:23.525562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:23.530664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27f882-8a39-4bdd-a5fd-e678d0ef6ad4 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:23.531102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27f882-8a39-4bdd-a5fd-e678d0ef6ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:23.531244 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27f882-8a39-4bdd-a5fd-e678d0ef6ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:23.605765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-eb27f882-8a39-4bdd-a5fd-e678d0ef6ad4 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:23.606989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-eb27f882-8a39-4bdd-a5fd-e678d0ef6ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:23.648218 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:23.703295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:06:23.853397 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7849608-48a3-4c59-b484-19a38fb7896f None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:23.854916 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f7849608-48a3-4c59-b484-19a38fb7896f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:23.998586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7849608-48a3-4c59-b484-19a38fb7896f None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:24.052476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:24.112887 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcf720-c895-4616-8e34-64b894488b3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:24.113474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcf720-c895-4616-8e34-64b894488b3e service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:24.113763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcf720-c895-4616-8e34-64b894488b3e service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:24.114041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcf720-c895-4616-8e34-64b894488b3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:24.115717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-effcf720-c895-4616-8e34-64b894488b3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:24.204906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcf720-c895-4616-8e34-64b894488b3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:24.206723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcf720-c895-4616-8e34-64b894488b3e service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:24.225023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 719/1437] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:25.026949 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f73c53-0ee7-4c08-89ba-90b1a5046a28 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:25.027459 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f73c53-0ee7-4c08-89ba-90b1a5046a28 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:25.027558 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f73c53-0ee7-4c08-89ba-90b1a5046a28 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:25.028869 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:25.034013 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789bfb2-98a2-4a68-a190-bd6e2f92684f None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:25.034561 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789bfb2-98a2-4a68-a190-bd6e2f92684f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:25.034972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789bfb2-98a2-4a68-a190-bd6e2f92684f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:25.109584 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-d789bfb2-98a2-4a68-a190-bd6e2f92684f None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:25.110852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d789bfb2-98a2-4a68-a190-bd6e2f92684f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:25.113044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:25.207565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:06:25.362972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f73c53-0ee7-4c08-89ba-90b1a5046a28 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:25.364489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-07f73c53-0ee7-4c08-89ba-90b1a5046a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:25.510921 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f73c53-0ee7-4c08-89ba-90b1a5046a28 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:25.537303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:25.626077 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b828a2-f08b-4880-8e25-cd3c3ec58cb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:25.626751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b828a2-f08b-4880-8e25-cd3c3ec58cb1 service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:25.627040 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b828a2-f08b-4880-8e25-cd3c3ec58cb1 service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:25.627319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b828a2-f08b-4880-8e25-cd3c3ec58cb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:25.629531 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-87b828a2-f08b-4880-8e25-cd3c3ec58cb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:25.717933 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b828a2-f08b-4880-8e25-cd3c3ec58cb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:25.719590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b828a2-f08b-4880-8e25-cd3c3ec58cb1 service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:25.738948 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 720/1440] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:27.678538 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429261b-426a-4149-9d5a-9686375d3b51 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:27.679057 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429261b-426a-4149-9d5a-9686375d3b51 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:27.679163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2429261b-426a-4149-9d5a-9686375d3b51 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:27.680516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:27.685735 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d13e-b0e4-4658-a0ee-d5275a0112d6 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:27.686158 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d13e-b0e4-4658-a0ee-d5275a0112d6 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:27.686333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d13e-b0e4-4658-a0ee-d5275a0112d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:27.727450 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:27.752552 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-e806d13e-b0e4-4658-a0ee-d5275a0112d6 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:27.754085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e806d13e-b0e4-4658-a0ee-d5275a0112d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:27.850318 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:06:27.888725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429261b-426a-4149-9d5a-9686375d3b51 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:27.890231 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2429261b-426a-4149-9d5a-9686375d3b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:27.993833 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2429261b-426a-4149-9d5a-9686375d3b51 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:28.026041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:28.067695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c4a43-3ac4-4280-a355-7ab088cd5a65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:28.068359 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c4a43-3ac4-4280-a355-7ab088cd5a65 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:28.068647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c4a43-3ac4-4280-a355-7ab088cd5a65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:28.068931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c4a43-3ac4-4280-a355-7ab088cd5a65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:28.070561 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d71c4a43-3ac4-4280-a355-7ab088cd5a65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:28.184058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c4a43-3ac4-4280-a355-7ab088cd5a65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:28.185488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c4a43-3ac4-4280-a355-7ab088cd5a65 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:28.203927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 722/1443] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:30.406862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f06cae-24b0-4224-ad23-40ad511854b6 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:30.407419 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f06cae-24b0-4224-ad23-40ad511854b6 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:30.407419 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f06cae-24b0-4224-ad23-40ad511854b6 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:30.409368 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:06:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:30.414750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc633a-8216-4b48-b7cf-471939e5de43 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:30.415168 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc633a-8216-4b48-b7cf-471939e5de43 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:30.415322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc633a-8216-4b48-b7cf-471939e5de43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:30.479550 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-a0bc633a-8216-4b48-b7cf-471939e5de43 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:30.480776 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-a0bc633a-8216-4b48-b7cf-471939e5de43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:30.494180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:30.578817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:06:30.619289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f06cae-24b0-4224-ad23-40ad511854b6 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:30.620795 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-79f06cae-24b0-4224-ad23-40ad511854b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:30.722971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f06cae-24b0-4224-ad23-40ad511854b6 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:30.734746 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:30.796472 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94c3b5-17ea-4428-8b01-e61dc8a231ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:30.797127 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94c3b5-17ea-4428-8b01-e61dc8a231ca service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:30.797415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94c3b5-17ea-4428-8b01-e61dc8a231ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:30.797697 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94c3b5-17ea-4428-8b01-e61dc8a231ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:30.800023 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0c94c3b5-17ea-4428-8b01-e61dc8a231ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:31.026576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94c3b5-17ea-4428-8b01-e61dc8a231ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:31.027954 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94c3b5-17ea-4428-8b01-e61dc8a231ca service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:31.046007 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 723/1446] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:33.448390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c9318-49db-41b9-856e-e22552052922 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:33.448980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c9318-49db-41b9-856e-e22552052922 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:33.448980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c9318-49db-41b9-856e-e22552052922 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:33.450326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:33.455593 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b901d-d99e-4fb9-9cbb-a07db9f76d9b None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:33.456008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b901d-d99e-4fb9-9cbb-a07db9f76d9b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:33.456131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b901d-d99e-4fb9-9cbb-a07db9f76d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:33.502181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:33.520048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-cd1b901d-d99e-4fb9-9cbb-a07db9f76d9b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:33.521245 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-cd1b901d-d99e-4fb9-9cbb-a07db9f76d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:33.619180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:06:33.660152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c9318-49db-41b9-856e-e22552052922 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:33.661660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-972c9318-49db-41b9-856e-e22552052922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:33.766013 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c9318-49db-41b9-856e-e22552052922 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:33.806853 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:33.841100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988125f-5233-4b00-81fc-90656cb13ac9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:33.841751 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988125f-5233-4b00-81fc-90656cb13ac9 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:33.842036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988125f-5233-4b00-81fc-90656cb13ac9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:33.842317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988125f-5233-4b00-81fc-90656cb13ac9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:33.843977 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c988125f-5233-4b00-81fc-90656cb13ac9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:33.952425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c988125f-5233-4b00-81fc-90656cb13ac9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:33.953855 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c988125f-5233-4b00-81fc-90656cb13ac9 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:33.972045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 725/1449] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:36.150269 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75468b61-0a36-41b7-b339-3c3171bc4470 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:36.150806 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75468b61-0a36-41b7-b339-3c3171bc4470 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:36.150937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75468b61-0a36-41b7-b339-3c3171bc4470 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:36.152241 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:36.157544 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384489b-c50a-4251-8374-640deb52baee None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:36.157981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384489b-c50a-4251-8374-640deb52baee None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:36.158109 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384489b-c50a-4251-8374-640deb52baee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:36.223865 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-6384489b-c50a-4251-8374-640deb52baee None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:36.225085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6384489b-c50a-4251-8374-640deb52baee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:36.246242 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:36.325446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:06:36.364953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75468b61-0a36-41b7-b339-3c3171bc4470 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:36.366501 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-75468b61-0a36-41b7-b339-3c3171bc4470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:36.469613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75468b61-0a36-41b7-b339-3c3171bc4470 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:36.490578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:36.544050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c50016-4c89-47a1-92c1-16d9827b77da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:36.544710 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c50016-4c89-47a1-92c1-16d9827b77da service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:36.544995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c50016-4c89-47a1-92c1-16d9827b77da service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:36.545273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c50016-4c89-47a1-92c1-16d9827b77da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:36.546945 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e7c50016-4c89-47a1-92c1-16d9827b77da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:36.655418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c50016-4c89-47a1-92c1-16d9827b77da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:36.656877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c50016-4c89-47a1-92c1-16d9827b77da service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:36.675834 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 726/1452] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:38.756495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24957799-6709-4690-ac66-81253d0ee0da None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:38.757022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24957799-6709-4690-ac66-81253d0ee0da None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:38.757022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24957799-6709-4690-ac66-81253d0ee0da None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:38.758352 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:38.763852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c348eb-4751-4793-ae47-ef8ceec2c07d None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:38.764285 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c348eb-4751-4793-ae47-ef8ceec2c07d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:38.764391 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c348eb-4751-4793-ae47-ef8ceec2c07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:38.824486 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:38.827020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-66c348eb-4751-4793-ae47-ef8ceec2c07d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:38.828785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-66c348eb-4751-4793-ae47-ef8ceec2c07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:38.923138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:06:38.962005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24957799-6709-4690-ac66-81253d0ee0da None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:38.963815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-24957799-6709-4690-ac66-81253d0ee0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:39.067839 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24957799-6709-4690-ac66-81253d0ee0da None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:39.117879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:39.143607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19873d9-767c-4473-b801-0b5f2a3d0bbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:39.144265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19873d9-767c-4473-b801-0b5f2a3d0bbf service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:39.144555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19873d9-767c-4473-b801-0b5f2a3d0bbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:39.144837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19873d9-767c-4473-b801-0b5f2a3d0bbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:39.146469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b19873d9-767c-4473-b801-0b5f2a3d0bbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:39.256277 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19873d9-767c-4473-b801-0b5f2a3d0bbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:39.257710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19873d9-767c-4473-b801-0b5f2a3d0bbf service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:39.276205 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 728/1455] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:41.398650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29186bf7-5823-401d-92f7-518a7168075b None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:41.399248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29186bf7-5823-401d-92f7-518a7168075b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:41.399248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29186bf7-5823-401d-92f7-518a7168075b None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:41.400629 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:41.405922 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbcdcd-bedd-43cb-a534-a9dd056d074c None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:41.406304 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbcdcd-bedd-43cb-a534-a9dd056d074c None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:41.406489 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbcdcd-bedd-43cb-a534-a9dd056d074c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:41.472951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-03bbcdcd-bedd-43cb-a534-a9dd056d074c None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:41.474171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-03bbcdcd-bedd-43cb-a534-a9dd056d074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:41.504434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:41.572401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:06:41.610848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29186bf7-5823-401d-92f7-518a7168075b None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:41.612973 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-29186bf7-5823-401d-92f7-518a7168075b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:41.720355 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29186bf7-5823-401d-92f7-518a7168075b None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:41.748995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:41.793935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682791dc-2707-4fee-ba8e-5658eaba192f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:41.794572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682791dc-2707-4fee-ba8e-5658eaba192f service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:41.794883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682791dc-2707-4fee-ba8e-5658eaba192f service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:41.795162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682791dc-2707-4fee-ba8e-5658eaba192f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:41.796763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-682791dc-2707-4fee-ba8e-5658eaba192f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:41.903282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682791dc-2707-4fee-ba8e-5658eaba192f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:41.904711 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682791dc-2707-4fee-ba8e-5658eaba192f service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:41.923322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 729/1458] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:44.074188 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fffd41-8885-4235-a16e-b27e691e1198 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:44.074613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fffd41-8885-4235-a16e-b27e691e1198 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:44.074810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fffd41-8885-4235-a16e-b27e691e1198 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:44.076170 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:44.081996 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c64798-e30e-43a2-945f-46c86b84016c None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:44.082424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c64798-e30e-43a2-945f-46c86b84016c None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:44.082545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c64798-e30e-43a2-945f-46c86b84016c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:44.146109 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-91c64798-e30e-43a2-945f-46c86b84016c None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:44.147339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-91c64798-e30e-43a2-945f-46c86b84016c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:44.155419 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:44.243162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:06:44.281167 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fffd41-8885-4235-a16e-b27e691e1198 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:44.282732 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d7fffd41-8885-4235-a16e-b27e691e1198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:44.387168 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fffd41-8885-4235-a16e-b27e691e1198 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:44.441766 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:44.460564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abeebf-f7eb-4319-a364-1465d6ba175d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:44.461212 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abeebf-f7eb-4319-a364-1465d6ba175d service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:44.461493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abeebf-f7eb-4319-a364-1465d6ba175d service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:44.461771 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abeebf-f7eb-4319-a364-1465d6ba175d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:44.463412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d0abeebf-f7eb-4319-a364-1465d6ba175d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:44.572037 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abeebf-f7eb-4319-a364-1465d6ba175d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:44.573481 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abeebf-f7eb-4319-a364-1465d6ba175d service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:44.592288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 731/1461] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:46.649032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28e505-d560-438f-83dd-d86db542fa5a None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:46.649582 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28e505-d560-438f-83dd-d86db542fa5a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:46.649582 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28e505-d560-438f-83dd-d86db542fa5a None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:46.650989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:46.656316 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a00db-cc16-40f6-a438-64d54435c1ba None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:46.656744 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a00db-cc16-40f6-a438-64d54435c1ba None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:46.656866 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a00db-cc16-40f6-a438-64d54435c1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:46.723664 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-976a00db-cc16-40f6-a438-64d54435c1ba None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:46.724891 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-976a00db-cc16-40f6-a438-64d54435c1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:46.764334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:46.825460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:06:46.865160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28e505-d560-438f-83dd-d86db542fa5a None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:46.866661 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5f28e505-d560-438f-83dd-d86db542fa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:46.968167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28e505-d560-438f-83dd-d86db542fa5a None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:47.007689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:47.043386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c02ef-9819-43fd-9800-bb18ee213be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:47.044032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c02ef-9819-43fd-9800-bb18ee213be1 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:47.044319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c02ef-9819-43fd-9800-bb18ee213be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:47.044598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c02ef-9819-43fd-9800-bb18ee213be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:47.046226 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-781c02ef-9819-43fd-9800-bb18ee213be1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:47.152246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c02ef-9819-43fd-9800-bb18ee213be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:47.153737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c02ef-9819-43fd-9800-bb18ee213be1 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:47.173875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 732/1464] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:49.237085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cff90-738d-4a0e-9962-3d1ae3a3dd9d None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:49.237659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cff90-738d-4a0e-9962-3d1ae3a3dd9d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:49.237659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cff90-738d-4a0e-9962-3d1ae3a3dd9d None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:49.239092 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:49.244407 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32581b7b-853e-48fc-93f3-3465f1341ea5 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:49.244807 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32581b7b-853e-48fc-93f3-3465f1341ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:49.244967 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32581b7b-853e-48fc-93f3-3465f1341ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:49.308670 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-32581b7b-853e-48fc-93f3-3465f1341ea5 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:49.309854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-32581b7b-853e-48fc-93f3-3465f1341ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:49.328080 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:49.407157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:06:49.444630 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cff90-738d-4a0e-9962-3d1ae3a3dd9d None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:49.446459 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8d8cff90-738d-4a0e-9962-3d1ae3a3dd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:49.674739 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8cff90-738d-4a0e-9962-3d1ae3a3dd9d None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:49.745594 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:49.752847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84d7d12-30ca-4255-a733-178afa1ab91b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:49.753501 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d7d12-30ca-4255-a733-178afa1ab91b service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:49.753778 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d7d12-30ca-4255-a733-178afa1ab91b service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:49.754051 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84d7d12-30ca-4255-a733-178afa1ab91b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:49.755708 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e84d7d12-30ca-4255-a733-178afa1ab91b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:49.861731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d7d12-30ca-4255-a733-178afa1ab91b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:49.863180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84d7d12-30ca-4255-a733-178afa1ab91b service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:49.880556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 734/1467] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:06:51.974446 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df257a-ed00-4640-8721-189c5c3476a8 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:51.975090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df257a-ed00-4640-8721-189c5c3476a8 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:51.975090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df257a-ed00-4640-8721-189c5c3476a8 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:51.976415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:06:51.981681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bdf1e-c8cb-4409-a60c-b75d4cd0d50b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:51.982085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bdf1e-c8cb-4409-a60c-b75d4cd0d50b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:51.982249 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bdf1e-c8cb-4409-a60c-b75d4cd0d50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:52.045717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-693bdf1e-c8cb-4409-a60c-b75d4cd0d50b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:06:52.046944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-693bdf1e-c8cb-4409-a60c-b75d4cd0d50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:52.095595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:52.145160 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:06:52.182298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df257a-ed00-4640-8721-189c5c3476a8 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:06:52.183841 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f1df257a-ed00-4640-8721-189c5c3476a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:52.288323 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df257a-ed00-4640-8721-189c5c3476a8 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:06:52.336893 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:06:52.363381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f1123-6932-4c93-a337-4111ac478f69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:06:52.364032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f1123-6932-4c93-a337-4111ac478f69 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:06:52.364314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f1123-6932-4c93-a337-4111ac478f69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:06:52.364619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f1123-6932-4c93-a337-4111ac478f69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:06:52.366252 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a96f1123-6932-4c93-a337-4111ac478f69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:06:52.474291 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f1123-6932-4c93-a337-4111ac478f69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:06:52.475758 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f1123-6932-4c93-a337-4111ac478f69 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:06:52.493685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 735/1470] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:07:02.142706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=22 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:07:02.491447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=66 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:07:12.247573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650d434-ff0f-4e90-85cf-e355799b3d82 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:12.248130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650d434-ff0f-4e90-85cf-e355799b3d82 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:12.248130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650d434-ff0f-4e90-85cf-e355799b3d82 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:12.249525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 736/1471] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:07:12.254735 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38e906-05c2-4455-8b35-a0cb3396c7eb None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:12.255160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38e906-05c2-4455-8b35-a0cb3396c7eb None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:12.255322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38e906-05c2-4455-8b35-a0cb3396c7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:12.321963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-bf38e906-05c2-4455-8b35-a0cb3396c7eb None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:07:12.323199 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-bf38e906-05c2-4455-8b35-a0cb3396c7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:12.419793 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 736/1472] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:07:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:07:12.502960 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cede020-2127-400c-b897-06bc92603202 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:12.503251 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cede020-2127-400c-b897-06bc92603202 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:12.503531 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cede020-2127-400c-b897-06bc92603202 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:12.504929 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 737/1473] 10.222.0.44 () {54 vars in 966 bytes} [Wed Jul 20 09:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:07:12.509731 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da850b-5561-45a2-afb0-2725aef03b6f None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:12.510017 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da850b-5561-45a2-afb0-2725aef03b6f None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:12.510296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da850b-5561-45a2-afb0-2725aef03b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:12.585003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a0da850b-5561-45a2-afb0-2725aef03b6f None None] MFA Rules not processed for user `bb93eb1d25c4410591125b0ab7a73149`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:07:12.586194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a0da850b-5561-45a2-afb0-2725aef03b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:12.606507 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:07:12.661767 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 737/1474] 10.222.0.44 () {58 vars in 1063 bytes} [Wed Jul 20 09:07:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:07:12.670285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cede020-2127-400c-b897-06bc92603202 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:07:12.671864 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8cede020-2127-400c-b897-06bc92603202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:12.772948 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cede020-2127-400c-b897-06bc92603202 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:07:12.843654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33d152-2067-4985-bf92-bcffe173f29f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb93eb1d25c4410591125b0ab7a73149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:07:12.844290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33d152-2067-4985-bf92-bcffe173f29f service glance] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:12.844568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33d152-2067-4985-bf92-bcffe173f29f service glance] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:12.845569 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33d152-2067-4985-bf92-bcffe173f29f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:12.847200 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-4d33d152-2067-4985-bf92-bcffe173f29f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:12.899364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:07:12.954604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33d152-2067-4985-bf92-bcffe173f29f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:07:12.956025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33d152-2067-4985-bf92-bcffe173f29f service glance] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:07:12.977971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 738/1475] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:07:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:07:15.108320 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df336107-d501-4732-a617-5575f203d71a None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:15.108915 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df336107-d501-4732-a617-5575f203d71a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:15.109029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df336107-d501-4732-a617-5575f203d71a None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:15.110447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 738/1476] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:07:15.116067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73435b5-1c86-4e2b-b232-8d88d776ca80 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:15.116514 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73435b5-1c86-4e2b-b232-8d88d776ca80 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:15.116685 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73435b5-1c86-4e2b-b232-8d88d776ca80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:15.180617 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-a73435b5-1c86-4e2b-b232-8d88d776ca80 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:07:15.181813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-a73435b5-1c86-4e2b-b232-8d88d776ca80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:15.288076 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 739/1477] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:07:15.313063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df336107-d501-4732-a617-5575f203d71a None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:07:15.314980 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-df336107-d501-4732-a617-5575f203d71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:15.476399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df336107-d501-4732-a617-5575f203d71a None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:07:15.531845 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:07:15.586311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b32a4e-38a6-4722-be40-d9815e26a7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:07:15.587005 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b32a4e-38a6-4722-be40-d9815e26a7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:15.587424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b32a4e-38a6-4722-be40-d9815e26a7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:15.587894 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b32a4e-38a6-4722-be40-d9815e26a7c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:15.589088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b32a4e-38a6-4722-be40-d9815e26a7c2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:07:15.590870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: WARNING py.warnings [None req-c5b32a4e-38a6-4722-be40-d9815e26a7c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:07:15.590870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: warnings.warn(msg) Jul 20 09:07:15.590870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: Jul 20 09:07:15.592003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b32a4e-38a6-4722-be40-d9815e26a7c2 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:07:15.613313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 739/1478] 10.222.0.44 () {56 vars in 1108 bytes} [Wed Jul 20 09:07:15 2022] GET /identity/v3/projects => generated 2553 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:07:17.709664 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e905c-f9fa-4bdf-aaaa-af1b8725fa9d None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:17.710219 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e905c-f9fa-4bdf-aaaa-af1b8725fa9d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:17.710219 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e905c-f9fa-4bdf-aaaa-af1b8725fa9d None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:17.711655 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 740/1479] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:07:17.717001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc8878-4688-4665-b5d2-753cd7c5b667 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:17.717446 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc8878-4688-4665-b5d2-753cd7c5b667 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:17.717575 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc8878-4688-4665-b5d2-753cd7c5b667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:17.781750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-46cc8878-4688-4665-b5d2-753cd7c5b667 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:07:17.783801 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-46cc8878-4688-4665-b5d2-753cd7c5b667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:17.882197 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 740/1480] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:07:17.926421 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e905c-f9fa-4bdf-aaaa-af1b8725fa9d None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:07:17.928029 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-743e905c-f9fa-4bdf-aaaa-af1b8725fa9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:17.939890 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:07:18.082507 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e905c-f9fa-4bdf-aaaa-af1b8725fa9d None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:07:18.165037 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:07:18.199914 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519eb2a-f199-4819-a28c-9421fa62b70d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:07:18.200568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519eb2a-f199-4819-a28c-9421fa62b70d service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:18.200849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519eb2a-f199-4819-a28c-9421fa62b70d service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:18.201127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519eb2a-f199-4819-a28c-9421fa62b70d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:18.202775 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f519eb2a-f199-4819-a28c-9421fa62b70d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:18.290963 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519eb2a-f199-4819-a28c-9421fa62b70d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:07:18.292541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519eb2a-f199-4819-a28c-9421fa62b70d service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:07:18.310891 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 741/1481] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:07:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:07:20.531034 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e87f34-aabe-4147-86b3-7cfb59f69589 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:20.531680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e87f34-aabe-4147-86b3-7cfb59f69589 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:20.531680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e87f34-aabe-4147-86b3-7cfb59f69589 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:20.533522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 741/1482] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:07:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:07:20.538800 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ed3fc-6d61-40da-82f0-e7834c4e9ad4 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:20.539217 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ed3fc-6d61-40da-82f0-e7834c4e9ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:20.539392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ed3fc-6d61-40da-82f0-e7834c4e9ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:20.604194 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-f57ed3fc-6d61-40da-82f0-e7834c4e9ad4 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:07:20.605323 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f57ed3fc-6d61-40da-82f0-e7834c4e9ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:20.652092 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:07:20.703897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 742/1483] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:07:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:07:20.741790 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e87f34-aabe-4147-86b3-7cfb59f69589 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:07:20.743355 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-35e87f34-aabe-4147-86b3-7cfb59f69589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:20.767091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:07:20.847005 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e87f34-aabe-4147-86b3-7cfb59f69589 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:07:20.918623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da33dc-1cad-4b6d-be87-150910d23bf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:07:20.919219 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da33dc-1cad-4b6d-be87-150910d23bf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:20.919503 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da33dc-1cad-4b6d-be87-150910d23bf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:20.919781 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da33dc-1cad-4b6d-be87-150910d23bf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:20.921396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-06da33dc-1cad-4b6d-be87-150910d23bf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:20.994510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:07:21.029390 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da33dc-1cad-4b6d-be87-150910d23bf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:07:21.030858 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da33dc-1cad-4b6d-be87-150910d23bf0 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:07:21.052172 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 742/1484] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:07:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:07:23.117847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a7958-f434-4c2c-9622-b9234067b537 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:23.118432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a7958-f434-4c2c-9622-b9234067b537 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:23.118432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a7958-f434-4c2c-9622-b9234067b537 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:23.119811 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 743/1485] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:07:23.124924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef640d47-929c-43f8-85a6-dabdf3f83304 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:23.125383 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef640d47-929c-43f8-85a6-dabdf3f83304 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:23.125519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef640d47-929c-43f8-85a6-dabdf3f83304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:23.192855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-ef640d47-929c-43f8-85a6-dabdf3f83304 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:07:23.194172 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ef640d47-929c-43f8-85a6-dabdf3f83304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:23.287598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 743/1486] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:07:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:07:23.328104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a7958-f434-4c2c-9622-b9234067b537 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:07:23.329692 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-202a7958-f434-4c2c-9622-b9234067b537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:23.433005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a7958-f434-4c2c-9622-b9234067b537 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:07:23.507211 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:07:23.509152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366627b-da22-4161-9cec-93815875c916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:07:23.509881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366627b-da22-4161-9cec-93815875c916 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:23.510395 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366627b-da22-4161-9cec-93815875c916 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:23.510719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366627b-da22-4161-9cec-93815875c916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:23.512738 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6366627b-da22-4161-9cec-93815875c916 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:23.621284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366627b-da22-4161-9cec-93815875c916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:07:23.622744 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366627b-da22-4161-9cec-93815875c916 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:07:23.640322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 744/1487] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:07:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:07:25.877838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5232d4-bc91-4f7f-bf60-f561ccd8e5ba None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:25.878401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5232d4-bc91-4f7f-bf60-f561ccd8e5ba None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:25.878401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5232d4-bc91-4f7f-bf60-f561ccd8e5ba None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:25.879813 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 744/1488] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:07:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:07:25.885082 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d381426-8e20-4cb7-9756-144dbfe124ea None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:25.885491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d381426-8e20-4cb7-9756-144dbfe124ea None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:25.885640 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d381426-8e20-4cb7-9756-144dbfe124ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:25.949109 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-0d381426-8e20-4cb7-9756-144dbfe124ea None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:07:25.950296 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-0d381426-8e20-4cb7-9756-144dbfe124ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:26.002858 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:07:26.045512 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 745/1489] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:07:26.087482 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5232d4-bc91-4f7f-bf60-f561ccd8e5ba None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:07:26.088979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-1b5232d4-bc91-4f7f-bf60-f561ccd8e5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:26.124320 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:07:26.193102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5232d4-bc91-4f7f-bf60-f561ccd8e5ba None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:07:26.264509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74203dc-c6e8-4858-8f00-23dc269c9b2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:07:26.265153 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74203dc-c6e8-4858-8f00-23dc269c9b2c service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:26.265432 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74203dc-c6e8-4858-8f00-23dc269c9b2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:26.265709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74203dc-c6e8-4858-8f00-23dc269c9b2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:26.268045 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d74203dc-c6e8-4858-8f00-23dc269c9b2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:26.345848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:07:26.374400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74203dc-c6e8-4858-8f00-23dc269c9b2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:07:26.375907 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74203dc-c6e8-4858-8f00-23dc269c9b2c service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:07:26.394429 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 745/1490] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:07:28.605181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4ea3e-c423-4837-a278-c5a55d5a0eef None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:28.605726 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4ea3e-c423-4837-a278-c5a55d5a0eef None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:28.605726 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4ea3e-c423-4837-a278-c5a55d5a0eef None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:28.607141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 746/1491] 10.222.0.44 () {54 vars in 926 bytes} [Wed Jul 20 09:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:07:28.612516 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7516b65-84c3-4226-a73b-a71626e53742 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:28.612916 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7516b65-84c3-4226-a73b-a71626e53742 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:28.613067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7516b65-84c3-4226-a73b-a71626e53742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:28.677282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-c7516b65-84c3-4226-a73b-a71626e53742 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:07:28.678470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c7516b65-84c3-4226-a73b-a71626e53742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:28.772419 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 746/1492] 10.222.0.44 () {58 vars in 1023 bytes} [Wed Jul 20 09:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:07:28.810183 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4ea3e-c423-4837-a278-c5a55d5a0eef None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:07:28.811713 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-01a4ea3e-c423-4837-a278-c5a55d5a0eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:28.912656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4ea3e-c423-4837-a278-c5a55d5a0eef None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:07:28.998282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebd99e-09da-4ab0-87e1-bc0293f3c5fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:07:28.998944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebd99e-09da-4ab0-87e1-bc0293f3c5fa service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:07:28.999224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebd99e-09da-4ab0-87e1-bc0293f3c5fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:07:28.999504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebd99e-09da-4ab0-87e1-bc0293f3c5fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:07:29.001128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8bebd99e-09da-4ab0-87e1-bc0293f3c5fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:07:29.009853 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:07:29.108087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bebd99e-09da-4ab0-87e1-bc0293f3c5fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:07:29.109503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bebd99e-09da-4ab0-87e1-bc0293f3c5fa service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:07:29.128048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 747/1493] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:07:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:07:38.770570 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=82 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:07:39.125210 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=50 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:08:55.716584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34af092-dc86-48a0-b374-f1b32ca75471 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:55.717649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34af092-dc86-48a0-b374-f1b32ca75471 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:55.717861 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34af092-dc86-48a0-b374-f1b32ca75471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:55.791149 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-d34af092-dc86-48a0-b374-f1b32ca75471 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:08:55.792354 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d34af092-dc86-48a0-b374-f1b32ca75471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:08:55.886994 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 747/1494] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:08:55.895787 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80583c55-bd96-4cf0-8828-00affb0d9c5c None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:55.896290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80583c55-bd96-4cf0-8828-00affb0d9c5c None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:55.896396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80583c55-bd96-4cf0-8828-00affb0d9c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:55.970025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-80583c55-bd96-4cf0-8828-00affb0d9c5c None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:08:55.971270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-80583c55-bd96-4cf0-8828-00affb0d9c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:08:56.065964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 748/1495] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:08:56.070736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34af092-dc86-48a0-b374-f1b32ca75471 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:08:56.072193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d34af092-dc86-48a0-b374-f1b32ca75471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:08:56.084181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34af092-dc86-48a0-b374-f1b32ca75471 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:08:56.145242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:08:56.181231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d2118-536d-4952-b142-77c8eae61cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:08:56.181902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d2118-536d-4952-b142-77c8eae61cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:56.182180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d2118-536d-4952-b142-77c8eae61cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:56.182453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d2118-536d-4952-b142-77c8eae61cb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:56.183210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d2118-536d-4952-b142-77c8eae61cb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:08:56.183517 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d2118-536d-4952-b142-77c8eae61cb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:08:56.184528 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: WARNING py.warnings [None req-912d2118-536d-4952-b142-77c8eae61cb9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:08:56.184528 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: warnings.warn(msg) Jul 20 09:08:56.184528 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: Jul 20 09:08:56.185409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d2118-536d-4952-b142-77c8eae61cb9 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:08:56.197534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 748/1496] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:08:56.205982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80583c55-bd96-4cf0-8828-00affb0d9c5c None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:08:56.215043 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80583c55-bd96-4cf0-8828-00affb0d9c5c None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:08:56.276510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:08:56.311518 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658294cb-91c6-4330-8e05-1d584d5973bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:08:56.312202 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658294cb-91c6-4330-8e05-1d584d5973bb admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:56.312493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658294cb-91c6-4330-8e05-1d584d5973bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:56.312769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658294cb-91c6-4330-8e05-1d584d5973bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:56.313469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658294cb-91c6-4330-8e05-1d584d5973bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:08:56.315230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: WARNING py.warnings [None req-658294cb-91c6-4330-8e05-1d584d5973bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:08:56.315230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: warnings.warn(msg) Jul 20 09:08:56.315230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: Jul 20 09:08:56.316422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658294cb-91c6-4330-8e05-1d584d5973bb admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:08:56.371600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 749/1497] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:08:56 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:08:56.376523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d2118-536d-4952-b142-77c8eae61cb9 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:08:56.378058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-912d2118-536d-4952-b142-77c8eae61cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:08:56.410064 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d2118-536d-4952-b142-77c8eae61cb9 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:08:56.502724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55425e7-8005-4ad3-8a24-58dd26d52953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:08:56.503392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55425e7-8005-4ad3-8a24-58dd26d52953 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:56.503674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55425e7-8005-4ad3-8a24-58dd26d52953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:56.503948 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55425e7-8005-4ad3-8a24-58dd26d52953 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:56.504637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55425e7-8005-4ad3-8a24-58dd26d52953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:08:56.505648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: WARNING py.warnings [None req-f55425e7-8005-4ad3-8a24-58dd26d52953 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:08:56.505648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: warnings.warn(msg) Jul 20 09:08:56.505648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: Jul 20 09:08:56.506852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55425e7-8005-4ad3-8a24-58dd26d52953 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:08:56.546960 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 749/1498] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:08:56 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:08:56.551173 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658294cb-91c6-4330-8e05-1d584d5973bb admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:08:56.552677 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-658294cb-91c6-4330-8e05-1d584d5973bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:08:56.615952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658294cb-91c6-4330-8e05-1d584d5973bb admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:08:56.700934 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:08:56.711639 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60724964-a442-4f06-9c72-95582306b7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:08:56.712305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60724964-a442-4f06-9c72-95582306b7ad admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:56.712584 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60724964-a442-4f06-9c72-95582306b7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:56.712856 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60724964-a442-4f06-9c72-95582306b7ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:56.713580 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60724964-a442-4f06-9c72-95582306b7ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:08:56.714576 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: WARNING py.warnings [None req-60724964-a442-4f06-9c72-95582306b7ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:08:56.714576 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: warnings.warn(msg) Jul 20 09:08:56.714576 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: Jul 20 09:08:56.715551 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60724964-a442-4f06-9c72-95582306b7ad admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:08:56.728435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 750/1499] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:08:56.732446 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55425e7-8005-4ad3-8a24-58dd26d52953 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:08:56.734713 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f55425e7-8005-4ad3-8a24-58dd26d52953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:08:56.746198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55425e7-8005-4ad3-8a24-58dd26d52953 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:08:56.772488 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:08:56.845214 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3063d-cd39-47c8-96b7-c0e3bf24184f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:08:56.845883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3063d-cd39-47c8-96b7-c0e3bf24184f admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:56.846166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3063d-cd39-47c8-96b7-c0e3bf24184f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:56.846440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3063d-cd39-47c8-96b7-c0e3bf24184f admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:56.847206 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3063d-cd39-47c8-96b7-c0e3bf24184f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:08:56.848226 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: WARNING py.warnings [None req-78e3063d-cd39-47c8-96b7-c0e3bf24184f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:08:56.848226 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: warnings.warn(msg) Jul 20 09:08:56.848226 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: Jul 20 09:08:56.849095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3063d-cd39-47c8-96b7-c0e3bf24184f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:08:56.861651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 750/1500] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:08:56.865810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60724964-a442-4f06-9c72-95582306b7ad admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:08:56.867370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-60724964-a442-4f06-9c72-95582306b7ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:08:56.900756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60724964-a442-4f06-9c72-95582306b7ad admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:08:56.993955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c1e72c-f148-458f-8dd5-0608a57fc3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:08:56.994786 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1e72c-f148-458f-8dd5-0608a57fc3ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:56.995076 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1e72c-f148-458f-8dd5-0608a57fc3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:56.995354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1e72c-f148-458f-8dd5-0608a57fc3ee admin admin] PATH_INFO: `/v3/projects/9ac9aa76c4b74fac814c413e7ccf8ab4/users/6a6daf1fc1244f6b96d799b763dcf088/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:57.018409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c1e72c-f148-458f-8dd5-0608a57fc3ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac9aa76c4b74fac814c413e7ccf8ab4, user_id=6a6daf1fc1244f6b96d799b763dcf088, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:08:57.019471 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: WARNING py.warnings [None req-b6c1e72c-f148-458f-8dd5-0608a57fc3ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:08:57.019471 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: warnings.warn(msg) Jul 20 09:08:57.019471 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: Jul 20 09:08:57.021388 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c1e72c-f148-458f-8dd5-0608a57fc3ee admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:08:57.046061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 751/1501] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:08:56 2022] PUT /identity/v3/projects/9ac9aa76c4b74fac814c413e7ccf8ab4/users/6a6daf1fc1244f6b96d799b763dcf088/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:08:57.058038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ab35b-60f4-4ecf-a37d-4e94e1d09448 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:57.058338 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ab35b-60f4-4ecf-a37d-4e94e1d09448 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:57.058623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ab35b-60f4-4ecf-a37d-4e94e1d09448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:57.114815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-533ab35b-60f4-4ecf-a37d-4e94e1d09448 None None] MFA Rules not processed for user `6a6daf1fc1244f6b96d799b763dcf088`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:08:57.116024 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-533ab35b-60f4-4ecf-a37d-4e94e1d09448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:08:57.192836 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 751/1502] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:08:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:08:57.206454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f087f64-ca0e-4f95-9630-06e9e9bb34c3 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:57.206781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f087f64-ca0e-4f95-9630-06e9e9bb34c3 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:57.207068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f087f64-ca0e-4f95-9630-06e9e9bb34c3 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:57.208479 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 752/1503] 10.222.0.44 () {54 vars in 964 bytes} [Wed Jul 20 09:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:08:57.213341 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a884065-561b-4ad1-9157-5365baf2966d None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:57.213639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a884065-561b-4ad1-9157-5365baf2966d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:57.213926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a884065-561b-4ad1-9157-5365baf2966d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:57.244753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:08:57.289952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-4a884065-561b-4ad1-9157-5365baf2966d None None] MFA Rules not processed for user `99dc3db93e9045b8b3ef59a030822972`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:08:57.291194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4a884065-561b-4ad1-9157-5365baf2966d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:08:57.367067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 752/1504] 10.222.0.44 () {58 vars in 1061 bytes} [Wed Jul 20 09:08:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:08:57.374909 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f087f64-ca0e-4f95-9630-06e9e9bb34c3 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:08:57.376391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5f087f64-ca0e-4f95-9630-06e9e9bb34c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:08:57.426111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:08:57.480651 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f087f64-ca0e-4f95-9630-06e9e9bb34c3 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:08:57.554224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f9214-79b5-4e70-8b2b-ed4a520184ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:08:57.554911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f9214-79b5-4e70-8b2b-ed4a520184ac service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:57.555197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f9214-79b5-4e70-8b2b-ed4a520184ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:57.555471 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f9214-79b5-4e70-8b2b-ed4a520184ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:57.557098 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b69f9214-79b5-4e70-8b2b-ed4a520184ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:08:57.654860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f9214-79b5-4e70-8b2b-ed4a520184ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:08:57.656981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f9214-79b5-4e70-8b2b-ed4a520184ac service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:08:57.660469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:08:57.675218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 753/1505] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:08:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:08:57.717117 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689aec5b-3484-486b-9a4c-1f42deb89401 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:57.717408 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689aec5b-3484-486b-9a4c-1f42deb89401 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:57.717725 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689aec5b-3484-486b-9a4c-1f42deb89401 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:57.719769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 753/1506] 10.222.0.44 () {52 vars in 829 bytes} [Wed Jul 20 09:08:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:08:57.723336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f9214-79b5-4e70-8b2b-ed4a520184ac service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:08:57.724540 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b69f9214-79b5-4e70-8b2b-ed4a520184ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:08:57.825365 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f9214-79b5-4e70-8b2b-ed4a520184ac service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:08:57.919790 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28d699-ab7c-45df-bea2-a480b3e21ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:08:57.920662 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28d699-ab7c-45df-bea2-a480b3e21ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:57.920944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28d699-ab7c-45df-bea2-a480b3e21ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:57.921253 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28d699-ab7c-45df-bea2-a480b3e21ac9 admin admin] PATH_INFO: `/v3/users/6a6daf1fc1244f6b96d799b763dcf088` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:57.933513 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28d699-ab7c-45df-bea2-a480b3e21ac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a6daf1fc1244f6b96d799b763dcf088)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:08:57.934535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: WARNING py.warnings [None req-4e28d699-ab7c-45df-bea2-a480b3e21ac9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:08:57.934535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: warnings.warn(msg) Jul 20 09:08:57.934535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: Jul 20 09:08:57.935779 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28d699-ab7c-45df-bea2-a480b3e21ac9 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:08:57.945955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:08:57.998069 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-4e28d699-ab7c-45df-bea2-a480b3e21ac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a6daf1fc1244f6b96d799b763dcf088'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:08:58.017230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-4e28d699-ab7c-45df-bea2-a480b3e21ac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a6daf1fc1244f6b96d799b763dcf088'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:08:58.028947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-4e28d699-ab7c-45df-bea2-a480b3e21ac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a6daf1fc1244f6b96d799b763dcf088'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:08:58.029498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-4e28d699-ab7c-45df-bea2-a480b3e21ac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a6daf1fc1244f6b96d799b763dcf088'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:08:58.038335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-4e28d699-ab7c-45df-bea2-a480b3e21ac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a6daf1fc1244f6b96d799b763dcf088'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:08:58.044072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 754/1507] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:08:57 2022] DELETE /identity/v3/users/6a6daf1fc1244f6b96d799b763dcf088 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:08:58.048619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13575003-e29f-4eeb-91e1-a5cc98d67728 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:58.049099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13575003-e29f-4eeb-91e1-a5cc98d67728 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:58.049206 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13575003-e29f-4eeb-91e1-a5cc98d67728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:58.103581 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-13575003-e29f-4eeb-91e1-a5cc98d67728 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:08:58.104776 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-13575003-e29f-4eeb-91e1-a5cc98d67728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:08:58.182192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:08:58.196614 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 754/1508] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:08:58.206945 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28d699-ab7c-45df-bea2-a480b3e21ac9 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:08:58.208162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-4e28d699-ab7c-45df-bea2-a480b3e21ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:08:58.309832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28d699-ab7c-45df-bea2-a480b3e21ac9 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:08:58.368569 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:08:58.390345 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23377cd0-44da-4622-b7c8-647211d9d877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:08:58.391055 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23377cd0-44da-4622-b7c8-647211d9d877 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:58.391334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23377cd0-44da-4622-b7c8-647211d9d877 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:58.391654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23377cd0-44da-4622-b7c8-647211d9d877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:58.393368 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-23377cd0-44da-4622-b7c8-647211d9d877 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:08:58.500086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23377cd0-44da-4622-b7c8-647211d9d877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:08:58.501571 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23377cd0-44da-4622-b7c8-647211d9d877 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:08:58.519853 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 755/1509] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:08:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:08:58.840793 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13575003-e29f-4eeb-91e1-a5cc98d67728 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:08:58.842050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-13575003-e29f-4eeb-91e1-a5cc98d67728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:08:58.854436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13575003-e29f-4eeb-91e1-a5cc98d67728 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:08:58.953033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c779e-f8dd-4439-8df0-cebd54fab5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:08:58.953830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c779e-f8dd-4439-8df0-cebd54fab5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:08:58.954110 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c779e-f8dd-4439-8df0-cebd54fab5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:08:58.954385 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c779e-f8dd-4439-8df0-cebd54fab5e6 admin admin] PATH_INFO: `/v3/projects/9ac9aa76c4b74fac814c413e7ccf8ab4` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:08:58.964067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c779e-f8dd-4439-8df0-cebd54fab5e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ac9aa76c4b74fac814c413e7ccf8ab4)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:08:58.965093 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: WARNING py.warnings [None req-0b3c779e-f8dd-4439-8df0-cebd54fab5e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:08:58.965093 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: warnings.warn(msg) Jul 20 09:08:58.965093 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: Jul 20 09:08:58.966284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c779e-f8dd-4439-8df0-cebd54fab5e6 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:08:59.023146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0b3c779e-f8dd-4439-8df0-cebd54fab5e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ac9aa76c4b74fac814c413e7ccf8ab4', 'request_id': 'req-0b3c779e-f8dd-4439-8df0-cebd54fab5e6'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:08:59.029022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0b3c779e-f8dd-4439-8df0-cebd54fab5e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ac9aa76c4b74fac814c413e7ccf8ab4', 'request_id': 'req-0b3c779e-f8dd-4439-8df0-cebd54fab5e6'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:08:59.038448 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0b3c779e-f8dd-4439-8df0-cebd54fab5e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ac9aa76c4b74fac814c413e7ccf8ab4', 'request_id': 'req-0b3c779e-f8dd-4439-8df0-cebd54fab5e6'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:08:59.072380 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0b3c779e-f8dd-4439-8df0-cebd54fab5e6 admin admin] The token cache is being invalidate because project 9ac9aa76c4b74fac814c413e7ccf8ab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:08:59.072983 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0b3c779e-f8dd-4439-8df0-cebd54fab5e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:08:59.073179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0b3c779e-f8dd-4439-8df0-cebd54fab5e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:08:59.074639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 755/1510] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:08:58 2022] DELETE /identity/v3/projects/9ac9aa76c4b74fac814c413e7ccf8ab4 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:09:08.517607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=62 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:09:09.070059 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:10:36.040362 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0e35f-a482-44a1-9f00-93ef61817bd1 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:36.040972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0e35f-a482-44a1-9f00-93ef61817bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:36.040972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0e35f-a482-44a1-9f00-93ef61817bd1 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:36.042309 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:10:36.047728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f876d-b3e7-48f5-bd86-701cec9a9048 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:36.048168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f876d-b3e7-48f5-bd86-701cec9a9048 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:36.048279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f876d-b3e7-48f5-bd86-701cec9a9048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:36.127716 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-664f876d-b3e7-48f5-bd86-701cec9a9048 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:10:36.128960 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-664f876d-b3e7-48f5-bd86-701cec9a9048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:10:36.231863 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:10:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:10:36.391245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0e35f-a482-44a1-9f00-93ef61817bd1 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:10:36.393309 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d1a0e35f-a482-44a1-9f00-93ef61817bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:10:36.510302 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0e35f-a482-44a1-9f00-93ef61817bd1 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:10:36.516544 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfe003-991e-488d-b540-7832dfcc8288 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:10:36.517196 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfe003-991e-488d-b540-7832dfcc8288 service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:36.517480 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfe003-991e-488d-b540-7832dfcc8288 service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:36.517761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfe003-991e-488d-b540-7832dfcc8288 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:36.525952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbfe003-991e-488d-b540-7832dfcc8288 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:10:36.527562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbfe003-991e-488d-b540-7832dfcc8288 service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:10:36.537027 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 757/1513] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:10:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:10:37.372930 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff5b21-6bcc-42e8-8630-73cc615d8c99 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:37.373513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff5b21-6bcc-42e8-8630-73cc615d8c99 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:37.373764 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff5b21-6bcc-42e8-8630-73cc615d8c99 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:37.375266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:10:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:10:37.380625 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4742979-418f-47a8-857c-fe8a15b0adce None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:37.381121 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4742979-418f-47a8-857c-fe8a15b0adce None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:37.381229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4742979-418f-47a8-857c-fe8a15b0adce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:37.401530 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-a4742979-418f-47a8-857c-fe8a15b0adce None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:10:37.402739 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-a4742979-418f-47a8-857c-fe8a15b0adce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:10:37.412106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:10:37.567308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff5b21-6bcc-42e8-8630-73cc615d8c99 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:10:37.583005 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff5b21-6bcc-42e8-8630-73cc615d8c99 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:10:37.588441 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7cca6-b417-4682-a06e-c59087cdf1f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:10:37.589077 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7cca6-b417-4682-a06e-c59087cdf1f2 service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:37.589364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7cca6-b417-4682-a06e-c59087cdf1f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:37.589645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7cca6-b417-4682-a06e-c59087cdf1f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:37.598412 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7cca6-b417-4682-a06e-c59087cdf1f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:10:37.600071 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7cca6-b417-4682-a06e-c59087cdf1f2 service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:10:37.609584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 758/1516] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:10:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:10:37.829523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5fd8e-6f5a-4059-a814-166efae9192f None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:37.829813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5fd8e-6f5a-4059-a814-166efae9192f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:37.830099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5fd8e-6f5a-4059-a814-166efae9192f None None] PATH_INFO: `/v3` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:37.831522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 759/1517] 10.222.0.74 () {54 vars in 925 bytes} [Wed Jul 20 09:10:37 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:10:37.837425 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4007-3aa0-4106-b8a1-3fb5572bba0a None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:37.837715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4007-3aa0-4106-b8a1-3fb5572bba0a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:37.837993 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4007-3aa0-4106-b8a1-3fb5572bba0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:37.896470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-bf9c4007-3aa0-4106-b8a1-3fb5572bba0a None None] MFA Rules not processed for user `066f69a87f5f48c7b20c1e0fcf5c1107`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:10:37.897585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-bf9c4007-3aa0-4106-b8a1-3fb5572bba0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:10:37.914001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:10:37.968899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 759/1518] 10.222.0.74 () {58 vars in 1015 bytes} [Wed Jul 20 09:10:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:10:41.758197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bd825-552f-477e-90da-e3baae66836d None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:41.758772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bd825-552f-477e-90da-e3baae66836d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:41.758772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bd825-552f-477e-90da-e3baae66836d None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:41.760267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 760/1519] 10.222.0.44 () {54 vars in 961 bytes} [Wed Jul 20 09:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:10:41.766503 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a8b77-910d-4af2-b7d5-eb23f70f8144 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:41.767074 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a8b77-910d-4af2-b7d5-eb23f70f8144 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:41.767186 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a8b77-910d-4af2-b7d5-eb23f70f8144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:41.785516 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-869a8b77-910d-4af2-b7d5-eb23f70f8144 None None] MFA Rules not processed for user `066f69a87f5f48c7b20c1e0fcf5c1107`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:10:41.786769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-869a8b77-910d-4af2-b7d5-eb23f70f8144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:10:41.795964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 760/1520] 10.222.0.44 () {58 vars in 1058 bytes} [Wed Jul 20 09:10:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:10:41.803694 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58bd825-552f-477e-90da-e3baae66836d None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:10:41.818255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58bd825-552f-477e-90da-e3baae66836d None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:10:41.823427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb532c-effb-4ee0-9f3c-3232e107aa1f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '066f69a87f5f48c7b20c1e0fcf5c1107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:10:41.824058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb532c-effb-4ee0-9f3c-3232e107aa1f service placement] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:41.824351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb532c-effb-4ee0-9f3c-3232e107aa1f service placement] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:41.824632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb532c-effb-4ee0-9f3c-3232e107aa1f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:41.833363 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb532c-effb-4ee0-9f3c-3232e107aa1f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:10:41.834975 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb532c-effb-4ee0-9f3c-3232e107aa1f service placement] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:10:41.844909 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 761/1521] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:10:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:10:48.847629 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d1baa-9697-4592-96f1-be4144813914 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:48.848160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d1baa-9697-4592-96f1-be4144813914 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:48.848160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d1baa-9697-4592-96f1-be4144813914 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:48.849469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:10:48.854297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f004c-f85a-4f01-b59b-990fed0b1cde None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:48.854721 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f004c-f85a-4f01-b59b-990fed0b1cde None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:48.854877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f004c-f85a-4f01-b59b-990fed0b1cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:48.872507 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-c19f004c-f85a-4f01-b59b-990fed0b1cde None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:10:48.874182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c19f004c-f85a-4f01-b59b-990fed0b1cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:10:48.882183 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:10:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:10:49.038034 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d1baa-9697-4592-96f1-be4144813914 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:10:49.052234 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d1baa-9697-4592-96f1-be4144813914 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:10:49.057509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52e859-cdef-4eaa-ac0d-c391103258f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:10:49.058133 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52e859-cdef-4eaa-ac0d-c391103258f2 service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:49.058412 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52e859-cdef-4eaa-ac0d-c391103258f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:49.058713 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52e859-cdef-4eaa-ac0d-c391103258f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:49.067304 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52e859-cdef-4eaa-ac0d-c391103258f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:10:49.068870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52e859-cdef-4eaa-ac0d-c391103258f2 service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:10:49.079494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 762/1524] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:10:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:10:49.895616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275c371-e04e-44cc-a672-23c1a9b372e6 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:49.896163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275c371-e04e-44cc-a672-23c1a9b372e6 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:49.896163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275c371-e04e-44cc-a672-23c1a9b372e6 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:49.897610 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 09:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:10:49.902420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432ed89-425b-489b-b2c1-c2553b21f749 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:49.902863 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432ed89-425b-489b-b2c1-c2553b21f749 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:49.903013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432ed89-425b-489b-b2c1-c2553b21f749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:49.921617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-1432ed89-425b-489b-b2c1-c2553b21f749 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:10:49.922829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-1432ed89-425b-489b-b2c1-c2553b21f749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:10:49.930566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 09:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:10:50.081934 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275c371-e04e-44cc-a672-23c1a9b372e6 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:10:50.096382 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275c371-e04e-44cc-a672-23c1a9b372e6 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:10:50.101714 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9822695-4791-4795-9e63-580318aa15d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:10:50.102343 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9822695-4791-4795-9e63-580318aa15d3 service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:50.102630 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9822695-4791-4795-9e63-580318aa15d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:50.102943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9822695-4791-4795-9e63-580318aa15d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:50.111822 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9822695-4791-4795-9e63-580318aa15d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:10:50.113396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9822695-4791-4795-9e63-580318aa15d3 service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:10:50.123496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 764/1527] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:10:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:10:54.268274 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfad40-145b-42f4-bfdb-abcd276db368 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:54.268750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfad40-145b-42f4-bfdb-abcd276db368 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:54.268864 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfad40-145b-42f4-bfdb-abcd276db368 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:54.270258 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:10:54.275465 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940f5a9-fec7-4250-8d07-4dcc8dcf942b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:54.275845 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940f5a9-fec7-4250-8d07-4dcc8dcf942b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:54.276051 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940f5a9-fec7-4250-8d07-4dcc8dcf942b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:54.293742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-b940f5a9-fec7-4250-8d07-4dcc8dcf942b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:10:54.295004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b940f5a9-fec7-4250-8d07-4dcc8dcf942b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:10:54.302788 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:10:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:10:54.377368 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfad40-145b-42f4-bfdb-abcd276db368 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:10:54.379080 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-79bfad40-145b-42f4-bfdb-abcd276db368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:10:54.457508 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfad40-145b-42f4-bfdb-abcd276db368 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:10:54.461020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a041d-b733-4410-86fa-c647c66c9bdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:10:54.461654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a041d-b733-4410-86fa-c647c66c9bdb service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:54.461936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a041d-b733-4410-86fa-c647c66c9bdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:54.462211 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a041d-b733-4410-86fa-c647c66c9bdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:54.471132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a041d-b733-4410-86fa-c647c66c9bdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:10:54.472500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a041d-b733-4410-86fa-c647c66c9bdb service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:10:54.481587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 765/1530] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:10:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:10:56.488611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414e34f-860d-4113-978d-3bf497e4fced None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:56.489142 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414e34f-860d-4113-978d-3bf497e4fced None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:56.489142 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414e34f-860d-4113-978d-3bf497e4fced None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:56.490468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:10:56.495675 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3255e5-6b0c-4feb-a97a-52a59a9a11e1 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:56.496100 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3255e5-6b0c-4feb-a97a-52a59a9a11e1 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:56.496313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3255e5-6b0c-4feb-a97a-52a59a9a11e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:56.514392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-4b3255e5-6b0c-4feb-a97a-52a59a9a11e1 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:10:56.515695 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4b3255e5-6b0c-4feb-a97a-52a59a9a11e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:10:56.523438 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:10:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:10:56.560199 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414e34f-860d-4113-978d-3bf497e4fced None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:10:56.570796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414e34f-860d-4113-978d-3bf497e4fced None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:10:56.574459 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d039da-2494-4eb3-91fd-04959088d20d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:10:56.575118 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d039da-2494-4eb3-91fd-04959088d20d service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:10:56.575398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d039da-2494-4eb3-91fd-04959088d20d service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:10:56.575679 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d039da-2494-4eb3-91fd-04959088d20d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:10:56.584340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d039da-2494-4eb3-91fd-04959088d20d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:10:56.585754 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d039da-2494-4eb3-91fd-04959088d20d service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:10:56.595533 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 767/1533] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:10:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:11:00.220490 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19904f89-8478-469f-ad77-d75fc40f92fd None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:11:00.220944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19904f89-8478-469f-ad77-d75fc40f92fd None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:11:00.221164 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19904f89-8478-469f-ad77-d75fc40f92fd None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:11:00.222412 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:11:00.227742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0e68b-f475-4796-97ca-6a624a8424d6 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:11:00.228178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0e68b-f475-4796-97ca-6a624a8424d6 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:11:00.228319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0e68b-f475-4796-97ca-6a624a8424d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:11:00.235273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=81 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:11:00.247735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-6fe0e68b-f475-4796-97ca-6a624a8424d6 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:11:00.248933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6fe0e68b-f475-4796-97ca-6a624a8424d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:11:00.256749 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:11:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:11:00.562663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19904f89-8478-469f-ad77-d75fc40f92fd None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:11:00.564355 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-19904f89-8478-469f-ad77-d75fc40f92fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:11:00.628256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:11:00.642080 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19904f89-8478-469f-ad77-d75fc40f92fd None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:11:00.645571 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70effe-670f-41b9-821d-f28b0e18085c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:11:00.646202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70effe-670f-41b9-821d-f28b0e18085c service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:11:00.646482 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70effe-670f-41b9-821d-f28b0e18085c service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:11:00.646783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70effe-670f-41b9-821d-f28b0e18085c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:11:00.656555 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70effe-670f-41b9-821d-f28b0e18085c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:11:00.657979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf70effe-670f-41b9-821d-f28b0e18085c service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:11:00.667251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 768/1536] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:11:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:11:02.729859 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22909ed8-1c4c-47a4-a43f-356d45a90476 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:11:02.730407 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22909ed8-1c4c-47a4-a43f-356d45a90476 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:11:02.730504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22909ed8-1c4c-47a4-a43f-356d45a90476 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:11:02.732537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 09:11:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:11:02.738140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fce0c-ae4b-4e1c-8f94-d10dca69bdca None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:11:02.738587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fce0c-ae4b-4e1c-8f94-d10dca69bdca None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:11:02.738774 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fce0c-ae4b-4e1c-8f94-d10dca69bdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:11:02.756462 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-381fce0c-ae4b-4e1c-8f94-d10dca69bdca None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:11:02.757644 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-381fce0c-ae4b-4e1c-8f94-d10dca69bdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:11:02.765853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 09:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:11:03.078466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22909ed8-1c4c-47a4-a43f-356d45a90476 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:11:03.080194 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-22909ed8-1c4c-47a4-a43f-356d45a90476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:11:03.093324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22909ed8-1c4c-47a4-a43f-356d45a90476 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:11:03.096844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa813dfb-6153-4923-a911-b22057045cd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:11:03.097482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa813dfb-6153-4923-a911-b22057045cd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:11:03.097771 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa813dfb-6153-4923-a911-b22057045cd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:11:03.098053 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa813dfb-6153-4923-a911-b22057045cd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:11:03.107427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa813dfb-6153-4923-a911-b22057045cd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:11:03.109005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa813dfb-6153-4923-a911-b22057045cd4 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:11:03.118317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 770/1539] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:11:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:11:12.751468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=11 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:11:13.114158 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=8 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:33.530954 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766e8b5-04f2-4012-88cf-7cd1ab8442b1 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:33.535206 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766e8b5-04f2-4012-88cf-7cd1ab8442b1 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:33.535206 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766e8b5-04f2-4012-88cf-7cd1ab8442b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:33.558890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-c766e8b5-04f2-4012-88cf-7cd1ab8442b1 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:33.560128 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c766e8b5-04f2-4012-88cf-7cd1ab8442b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:33.576173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 770/1540] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:33.585739 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ee123-1fab-4594-8a25-8db7ed6c8069 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:33.587142 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ee123-1fab-4594-8a25-8db7ed6c8069 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:33.587486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ee123-1fab-4594-8a25-8db7ed6c8069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:33.617209 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-636ee123-1fab-4594-8a25-8db7ed6c8069 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:33.618396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-636ee123-1fab-4594-8a25-8db7ed6c8069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:33.630620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 771/1541] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:33.635914 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766e8b5-04f2-4012-88cf-7cd1ab8442b1 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:33.646235 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766e8b5-04f2-4012-88cf-7cd1ab8442b1 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:33.651789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2d714-7d1f-4a4b-9a21-698c1275c1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:33.652582 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2d714-7d1f-4a4b-9a21-698c1275c1ad admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:33.652862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2d714-7d1f-4a4b-9a21-698c1275c1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:33.653134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2d714-7d1f-4a4b-9a21-698c1275c1ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:33.653994 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2d714-7d1f-4a4b-9a21-698c1275c1ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:12:33.654302 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2d714-7d1f-4a4b-9a21-698c1275c1ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:33.660021 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2d714-7d1f-4a4b-9a21-698c1275c1ad admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:33.674094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 771/1542] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:12:33.681994 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ee123-1fab-4594-8a25-8db7ed6c8069 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:33.698607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ee123-1fab-4594-8a25-8db7ed6c8069 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:33.703547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b61073-89b8-436b-b648-dd4c3af89453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:33.704223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b61073-89b8-436b-b648-dd4c3af89453 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:33.704504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b61073-89b8-436b-b648-dd4c3af89453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:33.705134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b61073-89b8-436b-b648-dd4c3af89453 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:33.705853 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b61073-89b8-436b-b648-dd4c3af89453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:33.707774 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b61073-89b8-436b-b648-dd4c3af89453 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:33.767831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 772/1543] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:33 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:33.772678 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2d714-7d1f-4a4b-9a21-698c1275c1ad admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:33.784615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2d714-7d1f-4a4b-9a21-698c1275c1ad admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:33.789163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5455d05-1616-4f0d-bb3c-8f93bf774709 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:33.789453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5455d05-1616-4f0d-bb3c-8f93bf774709 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:33.789730 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5455d05-1616-4f0d-bb3c-8f93bf774709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:33.820541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-f5455d05-1616-4f0d-bb3c-8f93bf774709 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:33.821748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f5455d05-1616-4f0d-bb3c-8f93bf774709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:33.897056 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 773/1544] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 20 09:12:33.898140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73632894-60d4-4a75-bf2e-82b7fc81db3e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:33.898431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73632894-60d4-4a75-bf2e-82b7fc81db3e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:33.901847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73632894-60d4-4a75-bf2e-82b7fc81db3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:33.923344 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7471b-5519-4760-bf30-6dff4d63708b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:33.924031 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7471b-5519-4760-bf30-6dff4d63708b admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:33.924313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7471b-5519-4760-bf30-6dff4d63708b admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:33.924594 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7471b-5519-4760-bf30-6dff4d63708b admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:33.925286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7471b-5519-4760-bf30-6dff4d63708b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:33.929564 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7471b-5519-4760-bf30-6dff4d63708b admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:33.943358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-73632894-60d4-4a75-bf2e-82b7fc81db3e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:33.944563 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-73632894-60d4-4a75-bf2e-82b7fc81db3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:33.957312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 774/1545] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:33.958895 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86f81a-3309-426d-88a8-c133cf7e1b94 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:33.959186 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86f81a-3309-426d-88a8-c133cf7e1b94 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:33.959467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86f81a-3309-426d-88a8-c133cf7e1b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:33.967357 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 772/1546] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:33 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:33.969018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5468b-a4c9-4d39-b25c-99bbfd7b20a2 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:33.969380 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5468b-a4c9-4d39-b25c-99bbfd7b20a2 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:33.969741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5468b-a4c9-4d39-b25c-99bbfd7b20a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:33.982178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-ea86f81a-3309-426d-88a8-c133cf7e1b94 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:33.983421 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ea86f81a-3309-426d-88a8-c133cf7e1b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:33.988933 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-62d5468b-a4c9-4d39-b25c-99bbfd7b20a2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:33.990178 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-62d5468b-a4c9-4d39-b25c-99bbfd7b20a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:33.992289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 775/1547] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:33.993674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b0666-f08d-4a32-9599-2afcb9b26857 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:33.993968 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b0666-f08d-4a32-9599-2afcb9b26857 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:33.994420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b0666-f08d-4a32-9599-2afcb9b26857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.001459 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 773/1548] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:34.003008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d953b-af83-4151-aa74-1e4bb1884e20 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.003792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d953b-af83-4151-aa74-1e4bb1884e20 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.004420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d953b-af83-4151-aa74-1e4bb1884e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.020909 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-451b0666-f08d-4a32-9599-2afcb9b26857 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:34.022108 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-451b0666-f08d-4a32-9599-2afcb9b26857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:34.024880 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-159d953b-af83-4151-aa74-1e4bb1884e20 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:34.026583 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-159d953b-af83-4151-aa74-1e4bb1884e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:34.031191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 776/1549] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:34.032533 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc681a8-f965-48b4-bf1c-9c4dcc8ae956 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.032827 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc681a8-f965-48b4-bf1c-9c4dcc8ae956 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.033106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc681a8-f965-48b4-bf1c-9c4dcc8ae956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.035103 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 774/1550] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:34.036266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8d7f2-59f1-4deb-b0a9-5e38d23db632 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.036558 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8d7f2-59f1-4deb-b0a9-5e38d23db632 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.036843 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8d7f2-59f1-4deb-b0a9-5e38d23db632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.051977 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-bbc681a8-f965-48b4-bf1c-9c4dcc8ae956 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:34.053167 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-bbc681a8-f965-48b4-bf1c-9c4dcc8ae956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:34.054908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a3f8d7f2-59f1-4deb-b0a9-5e38d23db632 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:34.056042 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a3f8d7f2-59f1-4deb-b0a9-5e38d23db632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:34.061136 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 777/1551] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:34.061902 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc681a8-f965-48b4-bf1c-9c4dcc8ae956 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.064547 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 775/1552] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:34.065849 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dac6b3-6913-4c42-bbd8-f887c4ae6eb6 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.066130 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dac6b3-6913-4c42-bbd8-f887c4ae6eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.066415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dac6b3-6913-4c42-bbd8-f887c4ae6eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.074041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc681a8-f965-48b4-bf1c-9c4dcc8ae956 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.078791 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80254c-fe34-447a-9a2b-493b5db4b92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.079457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80254c-fe34-447a-9a2b-493b5db4b92d admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.079742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80254c-fe34-447a-9a2b-493b5db4b92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.080018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80254c-fe34-447a-9a2b-493b5db4b92d admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.080753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80254c-fe34-447a-9a2b-493b5db4b92d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.082335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80254c-fe34-447a-9a2b-493b5db4b92d admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.085047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-10dac6b3-6913-4c42-bbd8-f887c4ae6eb6 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:34.086270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-10dac6b3-6913-4c42-bbd8-f887c4ae6eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:34.093334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 778/1553] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:34.094933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7e453-19cb-401d-a602-a95fa5da6f8f None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.095067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 776/1554] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:34.095171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7e453-19cb-401d-a602-a95fa5da6f8f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.095349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7e453-19cb-401d-a602-a95fa5da6f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.096891 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f39e2e-3d95-4e94-8928-31a8b2fa6a8a None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.097185 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f39e2e-3d95-4e94-8928-31a8b2fa6a8a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.097463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f39e2e-3d95-4e94-8928-31a8b2fa6a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.114899 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-e1c7e453-19cb-401d-a602-a95fa5da6f8f None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:34.116077 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-83f39e2e-3d95-4e94-8928-31a8b2fa6a8a None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:34.116184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e1c7e453-19cb-401d-a602-a95fa5da6f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:34.117238 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-83f39e2e-3d95-4e94-8928-31a8b2fa6a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:34.124320 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 779/1555] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:34.125181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7e453-19cb-401d-a602-a95fa5da6f8f None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.125610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 777/1556] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:34.126736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b35b5-c965-4ae6-a976-507b0acfce90 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.127027 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b35b5-c965-4ae6-a976-507b0acfce90 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.127303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b35b5-c965-4ae6-a976-507b0acfce90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.141039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7e453-19cb-401d-a602-a95fa5da6f8f None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.144922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-bb9b35b5-c965-4ae6-a976-507b0acfce90 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:34.145621 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b60df1-8f27-424d-b113-44ade203d090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.146116 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-bb9b35b5-c965-4ae6-a976-507b0acfce90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:34.146280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b60df1-8f27-424d-b113-44ade203d090 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.146566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b60df1-8f27-424d-b113-44ade203d090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.146874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b60df1-8f27-424d-b113-44ade203d090 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.147602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b60df1-8f27-424d-b113-44ade203d090 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:12:34.147908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b60df1-8f27-424d-b113-44ade203d090 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.148931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: WARNING py.warnings [None req-f7b60df1-8f27-424d-b113-44ade203d090 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:12:34.148931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: warnings.warn(msg) Jul 20 09:12:34.148931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: Jul 20 09:12:34.149786 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b60df1-8f27-424d-b113-44ade203d090 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.155286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 778/1557] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:34.156392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bd41c-1303-4ce4-8978-bce742b3f8aa None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.156690 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bd41c-1303-4ce4-8978-bce742b3f8aa None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.156963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bd41c-1303-4ce4-8978-bce742b3f8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.165206 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 780/1558] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:12:34.165206 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b60df1-8f27-424d-b113-44ade203d090 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.174709 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b60df1-8f27-424d-b113-44ade203d090 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.178481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-ce0bd41c-1303-4ce4-8978-bce742b3f8aa None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:34.179329 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be02b04-3f1b-4877-8e58-d243eccf5090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.179713 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ce0bd41c-1303-4ce4-8978-bce742b3f8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:34.180104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be02b04-3f1b-4877-8e58-d243eccf5090 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.180394 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be02b04-3f1b-4877-8e58-d243eccf5090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.180674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be02b04-3f1b-4877-8e58-d243eccf5090 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.181394 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be02b04-3f1b-4877-8e58-d243eccf5090 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:12:34.181720 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be02b04-3f1b-4877-8e58-d243eccf5090 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.183897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be02b04-3f1b-4877-8e58-d243eccf5090 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.190983 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 779/1559] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:34.191885 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bd41c-1303-4ce4-8978-bce742b3f8aa None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.200288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 781/1560] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:12:34.201351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be02b04-3f1b-4877-8e58-d243eccf5090 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.202535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bd41c-1303-4ce4-8978-bce742b3f8aa None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.207169 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97e31b-7e76-4bf6-9a3c-03d9cb841144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.207835 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97e31b-7e76-4bf6-9a3c-03d9cb841144 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.208122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97e31b-7e76-4bf6-9a3c-03d9cb841144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.208404 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97e31b-7e76-4bf6-9a3c-03d9cb841144 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.209136 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97e31b-7e76-4bf6-9a3c-03d9cb841144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.210752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97e31b-7e76-4bf6-9a3c-03d9cb841144 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.212050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be02b04-3f1b-4877-8e58-d243eccf5090 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.217092 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e33c30-8ab6-4949-aa2b-500d146b9cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.217744 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e33c30-8ab6-4949-aa2b-500d146b9cff admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.218028 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e33c30-8ab6-4949-aa2b-500d146b9cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.218310 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e33c30-8ab6-4949-aa2b-500d146b9cff admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.219061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e33c30-8ab6-4949-aa2b-500d146b9cff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:12:34.219374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e33c30-8ab6-4949-aa2b-500d146b9cff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.220944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e33c30-8ab6-4949-aa2b-500d146b9cff admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.224365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 780/1561] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:34.224365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97e31b-7e76-4bf6-9a3c-03d9cb841144 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.233306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97e31b-7e76-4bf6-9a3c-03d9cb841144 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.241301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 782/1562] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:12:34.241301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf50f08-8d11-434d-971f-30b526cd768d None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.241301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf50f08-8d11-434d-971f-30b526cd768d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.241301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf50f08-8d11-434d-971f-30b526cd768d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.241663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855f42c-33e3-4985-98f7-b212e11517b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.241663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855f42c-33e3-4985-98f7-b212e11517b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.241663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855f42c-33e3-4985-98f7-b212e11517b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.241663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855f42c-33e3-4985-98f7-b212e11517b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.241973 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855f42c-33e3-4985-98f7-b212e11517b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:12:34.241973 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855f42c-33e3-4985-98f7-b212e11517b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.241973 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855f42c-33e3-4985-98f7-b212e11517b0 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.262066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 781/1563] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:12:34.268373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-3bf50f08-8d11-434d-971f-30b526cd768d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:34.268507 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855f42c-33e3-4985-98f7-b212e11517b0 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.268605 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-3bf50f08-8d11-434d-971f-30b526cd768d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:34.268686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855f42c-33e3-4985-98f7-b212e11517b0 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.268776 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 783/1564] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:34.268776 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf50f08-8d11-434d-971f-30b526cd768d None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.271282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56035f-2456-4854-8634-e3adb99df21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.272124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56035f-2456-4854-8634-e3adb99df21c admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.272407 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56035f-2456-4854-8634-e3adb99df21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.272688 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56035f-2456-4854-8634-e3adb99df21c admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.273413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56035f-2456-4854-8634-e3adb99df21c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:12:34.273778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56035f-2456-4854-8634-e3adb99df21c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.275320 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56035f-2456-4854-8634-e3adb99df21c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.279825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf50f08-8d11-434d-971f-30b526cd768d None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.284645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d143c9-c2de-4a11-ba79-a9cd4e5c2aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.285357 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d143c9-c2de-4a11-ba79-a9cd4e5c2aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.285590 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d143c9-c2de-4a11-ba79-a9cd4e5c2aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.285871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d143c9-c2de-4a11-ba79-a9cd4e5c2aa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.286563 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d143c9-c2de-4a11-ba79-a9cd4e5c2aa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.287677 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 782/1565] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:12:34.288496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d143c9-c2de-4a11-ba79-a9cd4e5c2aa5 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.288598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56035f-2456-4854-8634-e3adb99df21c admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.300095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56035f-2456-4854-8634-e3adb99df21c admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.305062 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602eb7b-81a5-420a-9253-6a4d7f24c80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.305957 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602eb7b-81a5-420a-9253-6a4d7f24c80b admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.306290 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602eb7b-81a5-420a-9253-6a4d7f24c80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.306614 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602eb7b-81a5-420a-9253-6a4d7f24c80b admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.308290 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602eb7b-81a5-420a-9253-6a4d7f24c80b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:12:34.308643 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602eb7b-81a5-420a-9253-6a4d7f24c80b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.310322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602eb7b-81a5-420a-9253-6a4d7f24c80b admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.316582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 784/1566] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:34.317394 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d143c9-c2de-4a11-ba79-a9cd4e5c2aa5 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.323124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 783/1567] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:12:34.323925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602eb7b-81a5-420a-9253-6a4d7f24c80b admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.334589 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602eb7b-81a5-420a-9253-6a4d7f24c80b admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.337023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:34.367169 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:34.390612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d143c9-c2de-4a11-ba79-a9cd4e5c2aa5 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.395442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2d633-64f0-4e40-98bd-32fd3fe1d773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.395970 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e8af4-1643-4fcd-8bc8-5c3a884d8b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.396113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2d633-64f0-4e40-98bd-32fd3fe1d773 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.396384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2d633-64f0-4e40-98bd-32fd3fe1d773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.396693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2d633-64f0-4e40-98bd-32fd3fe1d773 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.396881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e8af4-1643-4fcd-8bc8-5c3a884d8b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.397291 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e8af4-1643-4fcd-8bc8-5c3a884d8b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.397394 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2d633-64f0-4e40-98bd-32fd3fe1d773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.397572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e8af4-1643-4fcd-8bc8-5c3a884d8b57 admin admin] PATH_INFO: `/v3/projects/f571b8b6410349418219f9d80d2df4fd/users/307f7fbfe4ff45daba698c930b1faff3/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.399314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2d633-64f0-4e40-98bd-32fd3fe1d773 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.413176 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e8af4-1643-4fcd-8bc8-5c3a884d8b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f571b8b6410349418219f9d80d2df4fd, user_id=307f7fbfe4ff45daba698c930b1faff3, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.414246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: WARNING py.warnings [None req-712e8af4-1643-4fcd-8bc8-5c3a884d8b57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:12:34.414246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: warnings.warn(msg) Jul 20 09:12:34.414246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: Jul 20 09:12:34.416283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e8af4-1643-4fcd-8bc8-5c3a884d8b57 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.429850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 785/1568] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:34.429850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2d633-64f0-4e40-98bd-32fd3fe1d773 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.433796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 784/1569] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:34 2022] PUT /identity/v3/projects/f571b8b6410349418219f9d80d2df4fd/users/307f7fbfe4ff45daba698c930b1faff3/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:34.434596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e8af4-1643-4fcd-8bc8-5c3a884d8b57 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.496821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e8af4-1643-4fcd-8bc8-5c3a884d8b57 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.501448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2d633-64f0-4e40-98bd-32fd3fe1d773 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.501601 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649b3fb-77f3-43c7-a0e1-466e024c130e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.502116 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649b3fb-77f3-43c7-a0e1-466e024c130e admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.502443 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649b3fb-77f3-43c7-a0e1-466e024c130e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.502667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649b3fb-77f3-43c7-a0e1-466e024c130e admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.503432 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649b3fb-77f3-43c7-a0e1-466e024c130e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.504381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: WARNING py.warnings [None req-3649b3fb-77f3-43c7-a0e1-466e024c130e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:12:34.504381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: warnings.warn(msg) Jul 20 09:12:34.504381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: Jul 20 09:12:34.505555 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649b3fb-77f3-43c7-a0e1-466e024c130e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.506632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46999aae-349e-43ed-922b-c81a553829d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.507309 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46999aae-349e-43ed-922b-c81a553829d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.507606 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46999aae-349e-43ed-922b-c81a553829d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.507869 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46999aae-349e-43ed-922b-c81a553829d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.508702 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46999aae-349e-43ed-922b-c81a553829d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.511462 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46999aae-349e-43ed-922b-c81a553829d9 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.531794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 785/1570] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:34.537254 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649b3fb-77f3-43c7-a0e1-466e024c130e admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.537402 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 786/1571] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:34.539535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46999aae-349e-43ed-922b-c81a553829d9 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.604261 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649b3fb-77f3-43c7-a0e1-466e024c130e admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.607058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46999aae-349e-43ed-922b-c81a553829d9 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.608879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9c6c4-0b60-46ed-921a-23a67dc33f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.609546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9c6c4-0b60-46ed-921a-23a67dc33f3f admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.609828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9c6c4-0b60-46ed-921a-23a67dc33f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.610111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9c6c4-0b60-46ed-921a-23a67dc33f3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.610892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9c6c4-0b60-46ed-921a-23a67dc33f3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:12:34.611202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9c6c4-0b60-46ed-921a-23a67dc33f3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.611744 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07098689-a4c0-41a7-9e78-0ed68734177e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.612406 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07098689-a4c0-41a7-9e78-0ed68734177e admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.612688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07098689-a4c0-41a7-9e78-0ed68734177e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.612807 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9c6c4-0b60-46ed-921a-23a67dc33f3f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.612967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07098689-a4c0-41a7-9e78-0ed68734177e admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.613648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07098689-a4c0-41a7-9e78-0ed68734177e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.615552 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07098689-a4c0-41a7-9e78-0ed68734177e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.625302 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 786/1572] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:12:34.626079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9c6c4-0b60-46ed-921a-23a67dc33f3f admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.637221 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9c6c4-0b60-46ed-921a-23a67dc33f3f admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.642013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29906ce0-ccd2-4258-8cca-16b1fc059365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.642667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29906ce0-ccd2-4258-8cca-16b1fc059365 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.642982 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29906ce0-ccd2-4258-8cca-16b1fc059365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.643256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29906ce0-ccd2-4258-8cca-16b1fc059365 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.643935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29906ce0-ccd2-4258-8cca-16b1fc059365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.645854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29906ce0-ccd2-4258-8cca-16b1fc059365 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.652324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 787/1573] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:34.652324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07098689-a4c0-41a7-9e78-0ed68734177e admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.661833 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:34.667491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 787/1574] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:34.668353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29906ce0-ccd2-4258-8cca-16b1fc059365 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.680027 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29906ce0-ccd2-4258-8cca-16b1fc059365 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.692201 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:34.719985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07098689-a4c0-41a7-9e78-0ed68734177e admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.724776 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a2c64-d15f-4510-8504-5b6bbdbb260d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.725437 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a2c64-d15f-4510-8504-5b6bbdbb260d admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.725718 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a2c64-d15f-4510-8504-5b6bbdbb260d admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.726002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a2c64-d15f-4510-8504-5b6bbdbb260d admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.726695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a2c64-d15f-4510-8504-5b6bbdbb260d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.729578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a2c64-d15f-4510-8504-5b6bbdbb260d admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.739039 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e65bbc-344c-4c6b-ad03-043fc67d1bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.739718 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e65bbc-344c-4c6b-ad03-043fc67d1bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.740034 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e65bbc-344c-4c6b-ad03-043fc67d1bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.740431 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e65bbc-344c-4c6b-ad03-043fc67d1bc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.741181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e65bbc-344c-4c6b-ad03-043fc67d1bc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.743144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e65bbc-344c-4c6b-ad03-043fc67d1bc5 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.757124 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 788/1575] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/projects => generated 393 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:34.758403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c272e1c-a318-4d0e-9844-2ac7f3d75327 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.758719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c272e1c-a318-4d0e-9844-2ac7f3d75327 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.759021 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c272e1c-a318-4d0e-9844-2ac7f3d75327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.765349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 788/1576] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:34.766209 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e65bbc-344c-4c6b-ad03-043fc67d1bc5 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.776381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e65bbc-344c-4c6b-ad03-043fc67d1bc5 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.788608 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-4c272e1c-a318-4d0e-9844-2ac7f3d75327 None None] MFA Rules not processed for user `307f7fbfe4ff45daba698c930b1faff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:34.789729 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-4c272e1c-a318-4d0e-9844-2ac7f3d75327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:34.832946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e52dac-5e54-41c3-aa08-177e1d0cee22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.833628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e52dac-5e54-41c3-aa08-177e1d0cee22 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.833943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e52dac-5e54-41c3-aa08-177e1d0cee22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.834223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e52dac-5e54-41c3-aa08-177e1d0cee22 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.834951 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e52dac-5e54-41c3-aa08-177e1d0cee22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.836868 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e52dac-5e54-41c3-aa08-177e1d0cee22 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.857171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 789/1577] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:34.858069 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c272e1c-a318-4d0e-9844-2ac7f3d75327 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.859522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 789/1578] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:34.860244 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e52dac-5e54-41c3-aa08-177e1d0cee22 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.870339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e52dac-5e54-41c3-aa08-177e1d0cee22 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.872072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c272e1c-a318-4d0e-9844-2ac7f3d75327 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.875019 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1d9f4-637d-4452-843a-1f8768b2c965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.875757 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1d9f4-637d-4452-843a-1f8768b2c965 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.876033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1d9f4-637d-4452-843a-1f8768b2c965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.876307 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1d9f4-637d-4452-843a-1f8768b2c965 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.876883 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f151b-38a5-46f9-8ccb-b8dc9672f263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.877028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1d9f4-637d-4452-843a-1f8768b2c965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.877551 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f151b-38a5-46f9-8ccb-b8dc9672f263 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.877833 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f151b-38a5-46f9-8ccb-b8dc9672f263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.878106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f151b-38a5-46f9-8ccb-b8dc9672f263 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.878823 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f151b-38a5-46f9-8ccb-b8dc9672f263 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.878940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1d9f4-637d-4452-843a-1f8768b2c965 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.879831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: WARNING py.warnings [None req-331f151b-38a5-46f9-8ccb-b8dc9672f263 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:12:34.879831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: warnings.warn(msg) Jul 20 09:12:34.879831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: Jul 20 09:12:34.881015 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f151b-38a5-46f9-8ccb-b8dc9672f263 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.907203 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 790/1579] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/users => generated 294 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:34.907203 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f151b-38a5-46f9-8ccb-b8dc9672f263 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.907521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 790/1580] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:34.907794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1d9f4-637d-4452-843a-1f8768b2c965 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:34.915773 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f151b-38a5-46f9-8ccb-b8dc9672f263 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.971935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228a236-7595-490d-9465-16857775d828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.972605 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228a236-7595-490d-9465-16857775d828 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.972888 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228a236-7595-490d-9465-16857775d828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.973166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228a236-7595-490d-9465-16857775d828 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.973848 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228a236-7595-490d-9465-16857775d828 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.975571 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1d9f4-637d-4452-843a-1f8768b2c965 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:34.975820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228a236-7595-490d-9465-16857775d828 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.981168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176b8fe-410e-427d-ae85-d9339a900150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:34.981831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176b8fe-410e-427d-ae85-d9339a900150 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:34.982108 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176b8fe-410e-427d-ae85-d9339a900150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:34.982380 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176b8fe-410e-427d-ae85-d9339a900150 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:34.983130 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176b8fe-410e-427d-ae85-d9339a900150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:34.984788 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176b8fe-410e-427d-ae85-d9339a900150 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:34.984943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:34.996782 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 791/1581] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:34.997836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 791/1582] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:34.998647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228a236-7595-490d-9465-16857775d828 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.000516 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176b8fe-410e-427d-ae85-d9339a900150 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.008072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228a236-7595-490d-9465-16857775d828 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.009684 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176b8fe-410e-427d-ae85-d9339a900150 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.013435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185bf3b-e9ca-4470-b7fa-d5992fa9b172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.014078 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185bf3b-e9ca-4470-b7fa-d5992fa9b172 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.014360 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185bf3b-e9ca-4470-b7fa-d5992fa9b172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.014638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185bf3b-e9ca-4470-b7fa-d5992fa9b172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.015266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859cf50-af26-49ee-b6cd-d018447d1f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.015409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185bf3b-e9ca-4470-b7fa-d5992fa9b172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.016256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859cf50-af26-49ee-b6cd-d018447d1f2d admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.016617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859cf50-af26-49ee-b6cd-d018447d1f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.016967 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859cf50-af26-49ee-b6cd-d018447d1f2d admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.017063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185bf3b-e9ca-4470-b7fa-d5992fa9b172 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.017781 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859cf50-af26-49ee-b6cd-d018447d1f2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.020403 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859cf50-af26-49ee-b6cd-d018447d1f2d admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.027547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 792/1583] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:35.028334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185bf3b-e9ca-4470-b7fa-d5992fa9b172 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.030518 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:35.039332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185bf3b-e9ca-4470-b7fa-d5992fa9b172 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.046084 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 792/1584] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:34 2022] POST /identity/v3/users => generated 297 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:35.046084 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5015ffe-c7b8-403f-a664-78efaf6b083b None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.046349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466a2d5-d47c-4f74-a63c-557ebdc4ecee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.046480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5015ffe-c7b8-403f-a664-78efaf6b083b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.046480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5015ffe-c7b8-403f-a664-78efaf6b083b None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.046706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466a2d5-d47c-4f74-a63c-557ebdc4ecee admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.046706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466a2d5-d47c-4f74-a63c-557ebdc4ecee admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.046706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466a2d5-d47c-4f74-a63c-557ebdc4ecee admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.046706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466a2d5-d47c-4f74-a63c-557ebdc4ecee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.047830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466a2d5-d47c-4f74-a63c-557ebdc4ecee admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.050876 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 793/1585] 10.222.0.44 () {54 vars in 962 bytes} [Wed Jul 20 09:12:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:12:35.051697 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5015ffe-c7b8-403f-a664-78efaf6b083b None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.062554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 793/1586] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:35.062554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466a2d5-d47c-4f74-a63c-557ebdc4ecee admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.065022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5015ffe-c7b8-403f-a664-78efaf6b083b None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.069372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466a2d5-d47c-4f74-a63c-557ebdc4ecee admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.069561 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf374c7e-9d37-41fd-8636-c023946bbe59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.070211 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf374c7e-9d37-41fd-8636-c023946bbe59 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.070522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf374c7e-9d37-41fd-8636-c023946bbe59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.070835 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf374c7e-9d37-41fd-8636-c023946bbe59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.071569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf374c7e-9d37-41fd-8636-c023946bbe59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.073552 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf374c7e-9d37-41fd-8636-c023946bbe59 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.075077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc92325-69cd-4346-bdcf-2f2c4cad556c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.075722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc92325-69cd-4346-bdcf-2f2c4cad556c admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.076004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc92325-69cd-4346-bdcf-2f2c4cad556c admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.076275 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc92325-69cd-4346-bdcf-2f2c4cad556c admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.076950 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc92325-69cd-4346-bdcf-2f2c4cad556c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.078846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc92325-69cd-4346-bdcf-2f2c4cad556c admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.084310 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 794/1587] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:35.085259 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf374c7e-9d37-41fd-8636-c023946bbe59 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.096050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf374c7e-9d37-41fd-8636-c023946bbe59 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.099065 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 794/1588] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:35 2022] POST /identity/v3/users => generated 302 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:35.099875 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc92325-69cd-4346-bdcf-2f2c4cad556c admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.101030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f851a51-161f-42bc-8135-c7ba5d72332a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.101686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f851a51-161f-42bc-8135-c7ba5d72332a admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.102078 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f851a51-161f-42bc-8135-c7ba5d72332a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.102332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f851a51-161f-42bc-8135-c7ba5d72332a admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.103165 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f851a51-161f-42bc-8135-c7ba5d72332a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.105961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f851a51-161f-42bc-8135-c7ba5d72332a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.110043 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc92325-69cd-4346-bdcf-2f2c4cad556c admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.114578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70defd-7ebf-46ed-88fc-7ca02dd69fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.115319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70defd-7ebf-46ed-88fc-7ca02dd69fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.115600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70defd-7ebf-46ed-88fc-7ca02dd69fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.115873 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70defd-7ebf-46ed-88fc-7ca02dd69fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.116312 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 795/1589] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:35.116593 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70defd-7ebf-46ed-88fc-7ca02dd69fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.117107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f851a51-161f-42bc-8135-c7ba5d72332a admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.118157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70defd-7ebf-46ed-88fc-7ca02dd69fb9 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.127753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f851a51-161f-42bc-8135-c7ba5d72332a admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.128802 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 795/1590] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:35.129608 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70defd-7ebf-46ed-88fc-7ca02dd69fb9 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.134816 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc8e55-027c-47c1-8269-54f59c69abff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.135501 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8e55-027c-47c1-8269-54f59c69abff admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.135824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8e55-027c-47c1-8269-54f59c69abff admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.136152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc8e55-027c-47c1-8269-54f59c69abff admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.136877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc8e55-027c-47c1-8269-54f59c69abff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.138464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc8e55-027c-47c1-8269-54f59c69abff admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.139772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70defd-7ebf-46ed-88fc-7ca02dd69fb9 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.144314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff642946-13e6-45b9-a073-f800e61a79cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.145236 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff642946-13e6-45b9-a073-f800e61a79cb admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.145529 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff642946-13e6-45b9-a073-f800e61a79cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.145805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff642946-13e6-45b9-a073-f800e61a79cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.146543 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff642946-13e6-45b9-a073-f800e61a79cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.148134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff642946-13e6-45b9-a073-f800e61a79cb admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.148938 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 796/1591] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:35.150293 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae086801-cd8f-4275-a874-93556952b5bf None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.150604 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae086801-cd8f-4275-a874-93556952b5bf None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.150907 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae086801-cd8f-4275-a874-93556952b5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.162004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 796/1592] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:35.162857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff642946-13e6-45b9-a073-f800e61a79cb admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.173809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff642946-13e6-45b9-a073-f800e61a79cb admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.178717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbb8b1-c29b-4881-9f22-a3e69c8dbd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.179479 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbb8b1-c29b-4881-9f22-a3e69c8dbd05 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.179767 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbb8b1-c29b-4881-9f22-a3e69c8dbd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.180063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbb8b1-c29b-4881-9f22-a3e69c8dbd05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.180790 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbb8b1-c29b-4881-9f22-a3e69c8dbd05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.183122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbb8b1-c29b-4881-9f22-a3e69c8dbd05 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.187776 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-ae086801-cd8f-4275-a874-93556952b5bf None None] MFA Rules not processed for user `be08c43f0d0c4a8194596e54500b44ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:35.188985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ae086801-cd8f-4275-a874-93556952b5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:35.193414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 797/1593] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:35.194347 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbb8b1-c29b-4881-9f22-a3e69c8dbd05 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.204486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbb8b1-c29b-4881-9f22-a3e69c8dbd05 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.208979 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec3f63-17ee-4a2b-94b6-5ebfdfbe8a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.209616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec3f63-17ee-4a2b-94b6-5ebfdfbe8a87 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.209899 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec3f63-17ee-4a2b-94b6-5ebfdfbe8a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.210176 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec3f63-17ee-4a2b-94b6-5ebfdfbe8a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.210922 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec3f63-17ee-4a2b-94b6-5ebfdfbe8a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.212484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec3f63-17ee-4a2b-94b6-5ebfdfbe8a87 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.222566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 798/1594] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:35.225964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec3f63-17ee-4a2b-94b6-5ebfdfbe8a87 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.235139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec3f63-17ee-4a2b-94b6-5ebfdfbe8a87 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.239785 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe19ee6-bc4b-41ad-8a2e-e1ff948e4f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.240425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe19ee6-bc4b-41ad-8a2e-e1ff948e4f15 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.240713 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe19ee6-bc4b-41ad-8a2e-e1ff948e4f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.240997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe19ee6-bc4b-41ad-8a2e-e1ff948e4f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.241718 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe19ee6-bc4b-41ad-8a2e-e1ff948e4f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.243319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe19ee6-bc4b-41ad-8a2e-e1ff948e4f15 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.254392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 799/1595] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:35.255013 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe19ee6-bc4b-41ad-8a2e-e1ff948e4f15 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.261824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 797/1596] 10.222.0.44 () {58 vars in 1059 bytes} [Wed Jul 20 09:12:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:35.262653 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae086801-cd8f-4275-a874-93556952b5bf None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.267654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe19ee6-bc4b-41ad-8a2e-e1ff948e4f15 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.273219 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae086801-cd8f-4275-a874-93556952b5bf None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.273697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559506dd-2e0b-41f0-a851-7aea0834ce6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.274591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559506dd-2e0b-41f0-a851-7aea0834ce6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.274908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559506dd-2e0b-41f0-a851-7aea0834ce6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.275189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559506dd-2e0b-41f0-a851-7aea0834ce6e admin admin] PATH_INFO: `/v3/projects/50baf84541494e7596c3d73a3214cb95/users/fc65968e84a8402ea0abee1f1749700c/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.278505 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88769ec7-970e-40ba-8aea-cb6caa223780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.279270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88769ec7-970e-40ba-8aea-cb6caa223780 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.279554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88769ec7-970e-40ba-8aea-cb6caa223780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.279824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88769ec7-970e-40ba-8aea-cb6caa223780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.280547 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88769ec7-970e-40ba-8aea-cb6caa223780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.282134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88769ec7-970e-40ba-8aea-cb6caa223780 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.292742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559506dd-2e0b-41f0-a851-7aea0834ce6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=50baf84541494e7596c3d73a3214cb95, user_id=fc65968e84a8402ea0abee1f1749700c, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.293112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 798/1597] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:35.293790 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88769ec7-970e-40ba-8aea-cb6caa223780 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.295483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559506dd-2e0b-41f0-a851-7aea0834ce6e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.303215 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88769ec7-970e-40ba-8aea-cb6caa223780 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.310862 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 800/1598] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:35 2022] PUT /identity/v3/projects/50baf84541494e7596c3d73a3214cb95/users/fc65968e84a8402ea0abee1f1749700c/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:35.319664 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559506dd-2e0b-41f0-a851-7aea0834ce6e admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.362818 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495c50d-a486-48a4-801f-afbfb7bf5bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.363758 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495c50d-a486-48a4-801f-afbfb7bf5bd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.364046 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495c50d-a486-48a4-801f-afbfb7bf5bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.364322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495c50d-a486-48a4-801f-afbfb7bf5bd2 admin admin] PATH_INFO: `/v3/projects/94fff07a1fa347b2b5ff853a2c4f1a1a/users/9a020687e95a40f88d521757d8b25619/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.379489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495c50d-a486-48a4-801f-afbfb7bf5bd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94fff07a1fa347b2b5ff853a2c4f1a1a, user_id=9a020687e95a40f88d521757d8b25619, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.382229 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495c50d-a486-48a4-801f-afbfb7bf5bd2 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.385917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559506dd-2e0b-41f0-a851-7aea0834ce6e admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.391284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27200604-420a-4e6f-8c6b-e8782aa87ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.392164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27200604-420a-4e6f-8c6b-e8782aa87ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.392449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27200604-420a-4e6f-8c6b-e8782aa87ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.392732 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27200604-420a-4e6f-8c6b-e8782aa87ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.393479 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27200604-420a-4e6f-8c6b-e8782aa87ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.395096 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27200604-420a-4e6f-8c6b-e8782aa87ce8 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.395247 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 799/1599] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:35 2022] PUT /identity/v3/projects/94fff07a1fa347b2b5ff853a2c4f1a1a/users/9a020687e95a40f88d521757d8b25619/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:35.398861 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495c50d-a486-48a4-801f-afbfb7bf5bd2 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.408993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 801/1600] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:35.409416 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:35.410218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27200604-420a-4e6f-8c6b-e8782aa87ce8 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.420483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27200604-420a-4e6f-8c6b-e8782aa87ce8 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.449751 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:35.464097 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495c50d-a486-48a4-801f-afbfb7bf5bd2 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.469084 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee99a20-9add-4946-9f7b-b427092050ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.469978 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee99a20-9add-4946-9f7b-b427092050ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.470258 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee99a20-9add-4946-9f7b-b427092050ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.470718 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee99a20-9add-4946-9f7b-b427092050ed admin admin] PATH_INFO: `/v3/projects/cf064c0854864e6ea1e96ae7dccdbfbc/users/43e7cbc8663c4d5bb4812dd818f744aa/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.480518 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c749e-adcc-4361-a75c-39d9785fa64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.481413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c749e-adcc-4361-a75c-39d9785fa64f admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.481695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c749e-adcc-4361-a75c-39d9785fa64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.481972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c749e-adcc-4361-a75c-39d9785fa64f admin admin] PATH_INFO: `/v3/projects/752497421a0b4c70a08bcca4393b8837/users/f0547075e10c498b8d13e1859197cea0/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.485717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee99a20-9add-4946-9f7b-b427092050ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf064c0854864e6ea1e96ae7dccdbfbc, user_id=43e7cbc8663c4d5bb4812dd818f744aa, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.488419 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee99a20-9add-4946-9f7b-b427092050ed admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.498233 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c749e-adcc-4361-a75c-39d9785fa64f admin admin] RBAC: Authorizing `identity:create_grant(project_id=752497421a0b4c70a08bcca4393b8837, user_id=f0547075e10c498b8d13e1859197cea0, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.501096 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c749e-adcc-4361-a75c-39d9785fa64f admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.502051 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 800/1601] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:35 2022] PUT /identity/v3/projects/cf064c0854864e6ea1e96ae7dccdbfbc/users/43e7cbc8663c4d5bb4812dd818f744aa/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:35.502960 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee99a20-9add-4946-9f7b-b427092050ed admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.516183 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 802/1602] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:35 2022] PUT /identity/v3/projects/752497421a0b4c70a08bcca4393b8837/users/f0547075e10c498b8d13e1859197cea0/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:35.517014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c749e-adcc-4361-a75c-39d9785fa64f admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.579718 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee99a20-9add-4946-9f7b-b427092050ed admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.593858 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c749e-adcc-4361-a75c-39d9785fa64f admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.599576 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe2766-1528-48e2-a1f6-52393ee733d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.600209 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe2766-1528-48e2-a1f6-52393ee733d3 service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.600487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe2766-1528-48e2-a1f6-52393ee733d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.600762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe2766-1528-48e2-a1f6-52393ee733d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.636146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8dd5c-d9b3-429a-b0ee-215c33ea0381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.636826 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8dd5c-d9b3-429a-b0ee-215c33ea0381 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.637113 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8dd5c-d9b3-429a-b0ee-215c33ea0381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.637394 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8dd5c-d9b3-429a-b0ee-215c33ea0381 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.638120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8dd5c-d9b3-429a-b0ee-215c33ea0381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.639758 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8dd5c-d9b3-429a-b0ee-215c33ea0381 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.649785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 801/1603] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:35.651110 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8dd5c-d9b3-429a-b0ee-215c33ea0381 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.660677 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8dd5c-d9b3-429a-b0ee-215c33ea0381 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.660904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe2766-1528-48e2-a1f6-52393ee733d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.662506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fe2766-1528-48e2-a1f6-52393ee733d3 service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.665378 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f948e-5a92-4ca1-92e6-33f31898a549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.666270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f948e-5a92-4ca1-92e6-33f31898a549 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.667113 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f948e-5a92-4ca1-92e6-33f31898a549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.667392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f948e-5a92-4ca1-92e6-33f31898a549 admin admin] PATH_INFO: `/v3/projects/e286919dacfc4db1a83308683c2a23ba/users/a24a58daa8ed4d77a649a670cc358638/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.672465 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 803/1604] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:35.673807 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ca05-85bc-42b3-8caa-ef356f0b698a None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.674106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ca05-85bc-42b3-8caa-ef356f0b698a None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.674418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2ca05-85bc-42b3-8caa-ef356f0b698a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.687629 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f948e-5a92-4ca1-92e6-33f31898a549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e286919dacfc4db1a83308683c2a23ba, user_id=a24a58daa8ed4d77a649a670cc358638, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.690433 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f948e-5a92-4ca1-92e6-33f31898a549 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.703648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-c0b2ca05-85bc-42b3-8caa-ef356f0b698a None None] MFA Rules not processed for user `fc65968e84a8402ea0abee1f1749700c`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:35.705304 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c0b2ca05-85bc-42b3-8caa-ef356f0b698a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:35.707664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 802/1605] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:35 2022] PUT /identity/v3/projects/e286919dacfc4db1a83308683c2a23ba/users/a24a58daa8ed4d77a649a670cc358638/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:35.709112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15916b-ea8f-46b1-8eeb-51d4d7fe95ac None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.709402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15916b-ea8f-46b1-8eeb-51d4d7fe95ac None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.709683 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15916b-ea8f-46b1-8eeb-51d4d7fe95ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.727870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-1f15916b-ea8f-46b1-8eeb-51d4d7fe95ac None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:35.729913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-1f15916b-ea8f-46b1-8eeb-51d4d7fe95ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:35.738531 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:35.760760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:35.790204 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 804/1606] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:35.791063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2ca05-85bc-42b3-8caa-ef356f0b698a None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.813029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 803/1607] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:35.814647 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446a212-71f4-4fa1-84b6-3ce2d2e5ca5e None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.814974 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446a212-71f4-4fa1-84b6-3ce2d2e5ca5e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.815376 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446a212-71f4-4fa1-84b6-3ce2d2e5ca5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.848993 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-d446a212-71f4-4fa1-84b6-3ce2d2e5ca5e None None] MFA Rules not processed for user `43e7cbc8663c4d5bb4812dd818f744aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:35.850644 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d446a212-71f4-4fa1-84b6-3ce2d2e5ca5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:35.882278 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2ca05-85bc-42b3-8caa-ef356f0b698a None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.887679 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c736403-5724-4b50-be4e-e338b7620b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.888590 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c736403-5724-4b50-be4e-e338b7620b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.888866 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c736403-5724-4b50-be4e-e338b7620b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.889147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c736403-5724-4b50-be4e-e338b7620b2a admin admin] PATH_INFO: `/v3/projects/19ee1fe6a5044166ba49d2a44e353f52/users/abdb339c5f5740d6aff4dfd1066f9384/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.905642 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c736403-5724-4b50-be4e-e338b7620b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ee1fe6a5044166ba49d2a44e353f52, user_id=abdb339c5f5740d6aff4dfd1066f9384, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:35.908796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c736403-5724-4b50-be4e-e338b7620b2a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:35.921751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 804/1608] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:35.923175 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbc6a3-4547-41f4-8c97-0e8775f7965a None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.923462 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbc6a3-4547-41f4-8c97-0e8775f7965a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.923739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbc6a3-4547-41f4-8c97-0e8775f7965a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:35.930343 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 805/1609] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:35 2022] PUT /identity/v3/projects/19ee1fe6a5044166ba49d2a44e353f52/users/abdb339c5f5740d6aff4dfd1066f9384/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:35.930343 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c736403-5724-4b50-be4e-e338b7620b2a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:35.942399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-c6bbc6a3-4547-41f4-8c97-0e8775f7965a None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:35.943612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c6bbc6a3-4547-41f4-8c97-0e8775f7965a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:35.988106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c736403-5724-4b50-be4e-e338b7620b2a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:35.992685 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61689e16-de73-4494-995c-408eb19d63b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:35.993573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61689e16-de73-4494-995c-408eb19d63b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:35.993855 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61689e16-de73-4494-995c-408eb19d63b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:35.994127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61689e16-de73-4494-995c-408eb19d63b0 admin admin] PATH_INFO: `/v3/projects/b9c72b43e6aa4911a1b441e47c193fab/users/8a58aef7a0bd43b1bc339369dfbca8c2/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.004262 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 805/1610] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:36.008134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5603c09-34e6-43eb-9a46-039461aa6d17 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:36.008426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5603c09-34e6-43eb-9a46-039461aa6d17 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:36.008705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5603c09-34e6-43eb-9a46-039461aa6d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.012834 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61689e16-de73-4494-995c-408eb19d63b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9c72b43e6aa4911a1b441e47c193fab, user_id=8a58aef7a0bd43b1bc339369dfbca8c2, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:36.015658 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61689e16-de73-4494-995c-408eb19d63b0 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:36.028946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a5603c09-34e6-43eb-9a46-039461aa6d17 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:36.029990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 806/1611] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:35 2022] PUT /identity/v3/projects/b9c72b43e6aa4911a1b441e47c193fab/users/8a58aef7a0bd43b1bc339369dfbca8c2/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:36.030296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a5603c09-34e6-43eb-9a46-039461aa6d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:36.030837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61689e16-de73-4494-995c-408eb19d63b0 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:36.071748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:36.087960 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:36.092817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 806/1612] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:36.093191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61689e16-de73-4494-995c-408eb19d63b0 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:36.094134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b0eef-a2e8-4fd2-9e5f-b09a1d9a0c87 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:36.094424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b0eef-a2e8-4fd2-9e5f-b09a1d9a0c87 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:36.094734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b0eef-a2e8-4fd2-9e5f-b09a1d9a0c87 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.099022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 807/1613] 10.222.0.44 () {54 vars in 962 bytes} [Wed Jul 20 09:12:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:12:36.100536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69172ee4-1bea-42d6-8a1e-f9a2f0db1e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:36.101196 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69172ee4-1bea-42d6-8a1e-f9a2f0db1e8d service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:36.101477 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69172ee4-1bea-42d6-8a1e-f9a2f0db1e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:36.101753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69172ee4-1bea-42d6-8a1e-f9a2f0db1e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.101865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65001e01-ec50-464c-a418-bae286e8fc6a None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:36.102294 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65001e01-ec50-464c-a418-bae286e8fc6a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:36.102698 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65001e01-ec50-464c-a418-bae286e8fc6a None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.104601 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 808/1614] 10.222.0.44 () {54 vars in 912 bytes} [Wed Jul 20 09:12:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:12:36.106080 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476e3b-fd4d-4621-8487-1a5949abb24e None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:36.106372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476e3b-fd4d-4621-8487-1a5949abb24e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:36.106654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476e3b-fd4d-4621-8487-1a5949abb24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.112720 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69172ee4-1bea-42d6-8a1e-f9a2f0db1e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:36.115075 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69172ee4-1bea-42d6-8a1e-f9a2f0db1e8d service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:36.124554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-83476e3b-fd4d-4621-8487-1a5949abb24e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:36.125747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-83476e3b-fd4d-4621-8487-1a5949abb24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:36.126248 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 807/1615] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:36.127092 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69172ee4-1bea-42d6-8a1e-f9a2f0db1e8d service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:36.136963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 809/1616] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:36.136963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83476e3b-fd4d-4621-8487-1a5949abb24e None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:36.146461 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83476e3b-fd4d-4621-8487-1a5949abb24e None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:36.151979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c257f-e14d-4db1-b2a0-7ce3df86a6e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:36.152633 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c257f-e14d-4db1-b2a0-7ce3df86a6e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:36.153103 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c257f-e14d-4db1-b2a0-7ce3df86a6e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:36.153605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c257f-e14d-4db1-b2a0-7ce3df86a6e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.163003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c257f-e14d-4db1-b2a0-7ce3df86a6e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:36.164385 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c257f-e14d-4db1-b2a0-7ce3df86a6e9 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:36.177924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 810/1617] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:36.179589 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb907b-b986-48a1-b46c-4eae2c5bc958 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:36.179878 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb907b-b986-48a1-b46c-4eae2c5bc958 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:36.180444 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb907b-b986-48a1-b46c-4eae2c5bc958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.193230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69172ee4-1bea-42d6-8a1e-f9a2f0db1e8d service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:36.197256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a11bd-5f50-448f-87a8-304fab18af2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:36.198227 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a11bd-5f50-448f-87a8-304fab18af2b service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:36.198476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a11bd-5f50-448f-87a8-304fab18af2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:36.198785 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a11bd-5f50-448f-87a8-304fab18af2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.209713 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-0cbb907b-b986-48a1-b46c-4eae2c5bc958 None None] MFA Rules not processed for user `8a58aef7a0bd43b1bc339369dfbca8c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:36.211125 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0cbb907b-b986-48a1-b46c-4eae2c5bc958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:36.269451 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a11bd-5f50-448f-87a8-304fab18af2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:36.276795 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a11bd-5f50-448f-87a8-304fab18af2b service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:36.288325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 811/1618] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:36.288325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab2801-970f-4055-a0d7-0822bde02869 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:36.288325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab2801-970f-4055-a0d7-0822bde02869 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:36.288325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab2801-970f-4055-a0d7-0822bde02869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.294319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 808/1619] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:36.295363 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a11bd-5f50-448f-87a8-304fab18af2b service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:36.306468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a11bd-5f50-448f-87a8-304fab18af2b service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:36.311969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399a1a6-729f-41ad-b830-d3a73ef39398 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:36.312889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399a1a6-729f-41ad-b830-d3a73ef39398 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:36.313270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399a1a6-729f-41ad-b830-d3a73ef39398 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:36.314725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399a1a6-729f-41ad-b830-d3a73ef39398 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.315691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-d5ab2801-970f-4055-a0d7-0822bde02869 None None] MFA Rules not processed for user `be08c43f0d0c4a8194596e54500b44ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:36.317385 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d5ab2801-970f-4055-a0d7-0822bde02869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:36.328226 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399a1a6-729f-41ad-b830-d3a73ef39398 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:36.331508 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399a1a6-729f-41ad-b830-d3a73ef39398 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:36.347303 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 809/1620] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:36.349066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5fc7e-8696-450d-937b-139e8ecb0e8f None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:36.359367 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5fc7e-8696-450d-937b-139e8ecb0e8f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:36.359824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5fc7e-8696-450d-937b-139e8ecb0e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.390543 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-27f5fc7e-8696-450d-937b-139e8ecb0e8f None None] MFA Rules not processed for user `be08c43f0d0c4a8194596e54500b44ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:36.392024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-27f5fc7e-8696-450d-937b-139e8ecb0e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:36.410637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 812/1621] 10.222.0.44 () {58 vars in 1059 bytes} [Wed Jul 20 09:12:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:36.411703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab2801-970f-4055-a0d7-0822bde02869 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:36.413119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d5ab2801-970f-4055-a0d7-0822bde02869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:36.425809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab2801-970f-4055-a0d7-0822bde02869 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:36.431504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd8a71-d408-4cf4-95f7-61ee2f11b146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:36.433397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd8a71-d408-4cf4-95f7-61ee2f11b146 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:36.434071 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd8a71-d408-4cf4-95f7-61ee2f11b146 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:36.434519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd8a71-d408-4cf4-95f7-61ee2f11b146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.445777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd8a71-d408-4cf4-95f7-61ee2f11b146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:36.448243 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd8a71-d408-4cf4-95f7-61ee2f11b146 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:36.459809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 813/1622] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:36.460632 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd8a71-d408-4cf4-95f7-61ee2f11b146 service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:36.471270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd8a71-d408-4cf4-95f7-61ee2f11b146 service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:36.476832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb215fdb-8a4a-4b2d-a75c-0926e57eb36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:36.478288 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb215fdb-8a4a-4b2d-a75c-0926e57eb36d admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:36.479320 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb215fdb-8a4a-4b2d-a75c-0926e57eb36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:36.479763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb215fdb-8a4a-4b2d-a75c-0926e57eb36d admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.480641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb215fdb-8a4a-4b2d-a75c-0926e57eb36d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:36.482875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb215fdb-8a4a-4b2d-a75c-0926e57eb36d admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:36.498091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 810/1623] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jul 20 09:12:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:36.499689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5fc7e-8696-450d-937b-139e8ecb0e8f None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:36.514257 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5fc7e-8696-450d-937b-139e8ecb0e8f None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:36.519699 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 814/1624] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:36 2022] POST /identity/v3/projects => generated 405 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:36.520984 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb215fdb-8a4a-4b2d-a75c-0926e57eb36d admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:36.524621 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1a17e-a388-4ba6-b744-a701aebc5d98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:36.525420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1a17e-a388-4ba6-b744-a701aebc5d98 service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:36.525778 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1a17e-a388-4ba6-b744-a701aebc5d98 service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:36.526125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1a17e-a388-4ba6-b744-a701aebc5d98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.542453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:36.558583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:36.633418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb215fdb-8a4a-4b2d-a75c-0926e57eb36d admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:36.646754 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255457d-77b7-4d08-bd9c-3e23314215fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:36.647437 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255457d-77b7-4d08-bd9c-3e23314215fd service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:36.647748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255457d-77b7-4d08-bd9c-3e23314215fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:36.648001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255457d-77b7-4d08-bd9c-3e23314215fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.654710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1a17e-a388-4ba6-b744-a701aebc5d98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:36.661697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1a17e-a388-4ba6-b744-a701aebc5d98 service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:36.696901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 811/1625] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:36.711233 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1a17e-a388-4ba6-b744-a701aebc5d98 service nova] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:36.749967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1a17e-a388-4ba6-b744-a701aebc5d98 service nova] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:36.860444 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255457d-77b7-4d08-bd9c-3e23314215fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:36.862020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255457d-77b7-4d08-bd9c-3e23314215fd service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:36.884477 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 815/1626] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:36.930515 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6712dc-ae00-4b59-af25-956f42748a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:36.932675 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6712dc-ae00-4b59-af25-956f42748a63 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:36.933014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6712dc-ae00-4b59-af25-956f42748a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:36.934458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6712dc-ae00-4b59-af25-956f42748a63 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:36.935759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6712dc-ae00-4b59-af25-956f42748a63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:36.939956 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6712dc-ae00-4b59-af25-956f42748a63 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:37.001618 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 812/1627] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:36 2022] POST /identity/v3/users => generated 309 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:37.002840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255457d-77b7-4d08-bd9c-3e23314215fd service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:37.030064 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255457d-77b7-4d08-bd9c-3e23314215fd service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:37.062882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81799d8-c849-4efa-9200-5a225b78c451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:37.063748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81799d8-c849-4efa-9200-5a225b78c451 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:37.063989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81799d8-c849-4efa-9200-5a225b78c451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:37.064260 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81799d8-c849-4efa-9200-5a225b78c451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:37.065453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81799d8-c849-4efa-9200-5a225b78c451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:37.074855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81799d8-c849-4efa-9200-5a225b78c451 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:37.125732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 816/1628] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:37.126788 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6712dc-ae00-4b59-af25-956f42748a63 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:37.138392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6712dc-ae00-4b59-af25-956f42748a63 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:37.158079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954e226-1153-4dc2-9653-3f5bcd4c8eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:37.158079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954e226-1153-4dc2-9653-3f5bcd4c8eab admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:37.158079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954e226-1153-4dc2-9653-3f5bcd4c8eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:37.158079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954e226-1153-4dc2-9653-3f5bcd4c8eab admin admin] PATH_INFO: `/v3/projects/360fdbda7e244170b6c0d1c183af3b6c/users/aeb96f2b075140ab917d005955d0156f/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:37.187145 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954e226-1153-4dc2-9653-3f5bcd4c8eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=360fdbda7e244170b6c0d1c183af3b6c, user_id=aeb96f2b075140ab917d005955d0156f, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:37.207872 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954e226-1153-4dc2-9653-3f5bcd4c8eab admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:37.242451 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 813/1629] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:37 2022] PUT /identity/v3/projects/360fdbda7e244170b6c0d1c183af3b6c/users/aeb96f2b075140ab917d005955d0156f/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:37.246259 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81799d8-c849-4efa-9200-5a225b78c451 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:37.268817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81799d8-c849-4efa-9200-5a225b78c451 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:37.406482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0097-4f57-4d37-88f9-08599654dae4 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:37.406861 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0097-4f57-4d37-88f9-08599654dae4 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:37.407086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0097-4f57-4d37-88f9-08599654dae4 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:37.409250 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 814/1630] 10.222.0.44 () {54 vars in 912 bytes} [Wed Jul 20 09:12:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:12:37.415208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513ac0e-0140-446e-80d2-fdf0227e9763 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:37.415404 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513ac0e-0140-446e-80d2-fdf0227e9763 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:37.415650 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513ac0e-0140-446e-80d2-fdf0227e9763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:37.437606 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3fd1c-e016-44e7-b861-57e0bd43f9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:37.438356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3fd1c-e016-44e7-b861-57e0bd43f9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:37.438646 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3fd1c-e016-44e7-b861-57e0bd43f9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:37.441782 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3fd1c-e016-44e7-b861-57e0bd43f9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:37.442605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3fd1c-e016-44e7-b861-57e0bd43f9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:37.446555 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3fd1c-e016-44e7-b861-57e0bd43f9ee admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:37.458967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-c513ac0e-0140-446e-80d2-fdf0227e9763 None None] MFA Rules not processed for user `be08c43f0d0c4a8194596e54500b44ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:37.460690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c513ac0e-0140-446e-80d2-fdf0227e9763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:37.481681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 817/1631] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:37.484935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3fd1c-e016-44e7-b861-57e0bd43f9ee admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:37.513897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3fd1c-e016-44e7-b861-57e0bd43f9ee admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:37.529326 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032f9f1-d435-4597-bb51-7fb1595966b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:37.532096 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032f9f1-d435-4597-bb51-7fb1595966b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:37.532385 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032f9f1-d435-4597-bb51-7fb1595966b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:37.535176 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032f9f1-d435-4597-bb51-7fb1595966b0 admin admin] PATH_INFO: `/v3/projects/360fdbda7e244170b6c0d1c183af3b6c/users/aeb96f2b075140ab917d005955d0156f/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:37.539334 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032f9f1-d435-4597-bb51-7fb1595966b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=360fdbda7e244170b6c0d1c183af3b6c, user_id=aeb96f2b075140ab917d005955d0156f, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:37.548612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032f9f1-d435-4597-bb51-7fb1595966b0 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:37.582448 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 818/1632] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:37 2022] PUT /identity/v3/projects/360fdbda7e244170b6c0d1c183af3b6c/users/aeb96f2b075140ab917d005955d0156f/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:37.593433 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb3629-31b1-454c-b2d0-4a55da62ed51 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:37.593725 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb3629-31b1-454c-b2d0-4a55da62ed51 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:37.594003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb3629-31b1-454c-b2d0-4a55da62ed51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:37.629920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-ccbb3629-31b1-454c-b2d0-4a55da62ed51 None None] MFA Rules not processed for user `aeb96f2b075140ab917d005955d0156f`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:37.631203 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 815/1633] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jul 20 09:12:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:12:37.631650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ccbb3629-31b1-454c-b2d0-4a55da62ed51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:37.647633 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513ac0e-0140-446e-80d2-fdf0227e9763 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:37.648613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:37.692042 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:37.757596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 819/1634] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:37 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:12:37.779241 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513ac0e-0140-446e-80d2-fdf0227e9763 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:37.831204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb3629-31b1-454c-b2d0-4a55da62ed51 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:37.906284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ae2a1-0f6b-416a-ae77-e6c321e243c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:37.906971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ae2a1-0f6b-416a-ae77-e6c321e243c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:37.907607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ae2a1-0f6b-416a-ae77-e6c321e243c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:37.907980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ae2a1-0f6b-416a-ae77-e6c321e243c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:37.925235 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb3629-31b1-454c-b2d0-4a55da62ed51 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:37.935475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a914b-2ced-4820-8e45-be4a02870813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:37.935475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a914b-2ced-4820-8e45-be4a02870813 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:37.935475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a914b-2ced-4820-8e45-be4a02870813 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:37.935475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a914b-2ced-4820-8e45-be4a02870813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:37.946296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a914b-2ced-4820-8e45-be4a02870813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:37.948936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a914b-2ced-4820-8e45-be4a02870813 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:37.972549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 820/1635] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:37 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:38.079425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ae2a1-0f6b-416a-ae77-e6c321e243c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:38.084451 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ae2a1-0f6b-416a-ae77-e6c321e243c1 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:38.127399 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 816/1636] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:12:38.347169 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a914b-2ced-4820-8e45-be4a02870813 service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:38.469879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a914b-2ced-4820-8e45-be4a02870813 service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:38.494559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ae2a1-0f6b-416a-ae77-e6c321e243c1 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:38.496813 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd15e9-b6a1-4694-aa70-cc895e80eac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:38.497714 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd15e9-b6a1-4694-aa70-cc895e80eac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:38.497999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd15e9-b6a1-4694-aa70-cc895e80eac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:38.498275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd15e9-b6a1-4694-aa70-cc895e80eac5 admin admin] PATH_INFO: `/v3/users/8a58aef7a0bd43b1bc339369dfbca8c2` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:38.501248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd15e9-b6a1-4694-aa70-cc895e80eac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a58aef7a0bd43b1bc339369dfbca8c2)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:38.502265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: WARNING py.warnings [None req-91fd15e9-b6a1-4694-aa70-cc895e80eac5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:12:38.502265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: warnings.warn(msg) Jul 20 09:12:38.502265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: Jul 20 09:12:38.503514 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd15e9-b6a1-4694-aa70-cc895e80eac5 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:38.521370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ae2a1-0f6b-416a-ae77-e6c321e243c1 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:38.539066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d04907-d6c0-4561-a25e-f2d342488b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:38.539758 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d04907-d6c0-4561-a25e-f2d342488b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:38.539897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d04907-d6c0-4561-a25e-f2d342488b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:38.542402 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d04907-d6c0-4561-a25e-f2d342488b4c admin admin] PATH_INFO: `/v3/users/fc65968e84a8402ea0abee1f1749700c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:38.544352 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d04907-d6c0-4561-a25e-f2d342488b4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc65968e84a8402ea0abee1f1749700c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:38.550954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d04907-d6c0-4561-a25e-f2d342488b4c admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:38.640909 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-39d04907-d6c0-4561-a25e-f2d342488b4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc65968e84a8402ea0abee1f1749700c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:38.652212 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-91fd15e9-b6a1-4694-aa70-cc895e80eac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a58aef7a0bd43b1bc339369dfbca8c2'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:38.667998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-39d04907-d6c0-4561-a25e-f2d342488b4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc65968e84a8402ea0abee1f1749700c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:38.673000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:38.677079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:38.696877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-39d04907-d6c0-4561-a25e-f2d342488b4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc65968e84a8402ea0abee1f1749700c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:38.699945 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-39d04907-d6c0-4561-a25e-f2d342488b4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc65968e84a8402ea0abee1f1749700c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:38.713888 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-91fd15e9-b6a1-4694-aa70-cc895e80eac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a58aef7a0bd43b1bc339369dfbca8c2'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:38.724508 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-39d04907-d6c0-4561-a25e-f2d342488b4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc65968e84a8402ea0abee1f1749700c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:38.748352 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 817/1637] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:12:38 2022] DELETE /identity/v3/users/fc65968e84a8402ea0abee1f1749700c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:38.756944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88469712-a4e8-40ff-91df-cafc29753980 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:38.756944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88469712-a4e8-40ff-91df-cafc29753980 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:38.756944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88469712-a4e8-40ff-91df-cafc29753980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:38.772584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-91fd15e9-b6a1-4694-aa70-cc895e80eac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a58aef7a0bd43b1bc339369dfbca8c2'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:38.776604 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-91fd15e9-b6a1-4694-aa70-cc895e80eac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a58aef7a0bd43b1bc339369dfbca8c2'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:38.799225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-91fd15e9-b6a1-4694-aa70-cc895e80eac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a58aef7a0bd43b1bc339369dfbca8c2'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:38.809933 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 821/1638] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:12:38 2022] DELETE /identity/v3/users/8a58aef7a0bd43b1bc339369dfbca8c2 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:38.815948 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19e197-f22d-4c33-af41-8a10045b7aed None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:38.816529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19e197-f22d-4c33-af41-8a10045b7aed None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:38.817545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19e197-f22d-4c33-af41-8a10045b7aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:38.820132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-88469712-a4e8-40ff-91df-cafc29753980 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:38.823814 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-88469712-a4e8-40ff-91df-cafc29753980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:38.866551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-8b19e197-f22d-4c33-af41-8a10045b7aed None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:38.868669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8b19e197-f22d-4c33-af41-8a10045b7aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:38.997976 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 818/1639] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:39.002716 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 822/1640] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:39.040626 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88469712-a4e8-40ff-91df-cafc29753980 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:39.042832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-88469712-a4e8-40ff-91df-cafc29753980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:39.044509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19e197-f22d-4c33-af41-8a10045b7aed None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:39.045894 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8b19e197-f22d-4c33-af41-8a10045b7aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:39.155120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88469712-a4e8-40ff-91df-cafc29753980 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:39.168862 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91c110-473e-4306-9080-9d8612e78eb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:39.171849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91c110-473e-4306-9080-9d8612e78eb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:39.172897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91c110-473e-4306-9080-9d8612e78eb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:39.173671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91c110-473e-4306-9080-9d8612e78eb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:39.187067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19e197-f22d-4c33-af41-8a10045b7aed None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:39.194424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8aef8-72ee-4ec1-b522-73f1a0387172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:39.195216 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8aef8-72ee-4ec1-b522-73f1a0387172 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:39.195936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8aef8-72ee-4ec1-b522-73f1a0387172 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:39.195936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8aef8-72ee-4ec1-b522-73f1a0387172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:39.202868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91c110-473e-4306-9080-9d8612e78eb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:39.204311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91c110-473e-4306-9080-9d8612e78eb3 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:39.220688 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8aef8-72ee-4ec1-b522-73f1a0387172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:39.220914 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 819/1641] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:39.222593 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8aef8-72ee-4ec1-b522-73f1a0387172 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:39.236168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 823/1642] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:12:40.577164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91c110-473e-4306-9080-9d8612e78eb3 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:40.578742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-eb91c110-473e-4306-9080-9d8612e78eb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:40.604526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91c110-473e-4306-9080-9d8612e78eb3 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:40.612019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468738f-2577-4ba8-a749-91eecfa478c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:40.613012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468738f-2577-4ba8-a749-91eecfa478c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:40.613365 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468738f-2577-4ba8-a749-91eecfa478c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:40.613711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468738f-2577-4ba8-a749-91eecfa478c6 admin admin] PATH_INFO: `/v3/projects/b9c72b43e6aa4911a1b441e47c193fab` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:40.615122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468738f-2577-4ba8-a749-91eecfa478c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9c72b43e6aa4911a1b441e47c193fab)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:40.616355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: WARNING py.warnings [None req-c468738f-2577-4ba8-a749-91eecfa478c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 09:12:40.616355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: warnings.warn(msg) Jul 20 09:12:40.616355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: Jul 20 09:12:40.617722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468738f-2577-4ba8-a749-91eecfa478c6 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:40.680991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8aef8-72ee-4ec1-b522-73f1a0387172 service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:40.683400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-eee8aef8-72ee-4ec1-b522-73f1a0387172 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:40.710171 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8aef8-72ee-4ec1-b522-73f1a0387172 service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:40.754625 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c468738f-2577-4ba8-a749-91eecfa478c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9c72b43e6aa4911a1b441e47c193fab', 'request_id': 'req-c468738f-2577-4ba8-a749-91eecfa478c6'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:40.770125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c468738f-2577-4ba8-a749-91eecfa478c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9c72b43e6aa4911a1b441e47c193fab', 'request_id': 'req-c468738f-2577-4ba8-a749-91eecfa478c6'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:40.817657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c468738f-2577-4ba8-a749-91eecfa478c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9c72b43e6aa4911a1b441e47c193fab', 'request_id': 'req-c468738f-2577-4ba8-a749-91eecfa478c6'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:40.819938 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a422ec92-4c30-496b-bceb-04f3849373e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:40.821037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422ec92-4c30-496b-bceb-04f3849373e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:40.821371 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422ec92-4c30-496b-bceb-04f3849373e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:40.821656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422ec92-4c30-496b-bceb-04f3849373e8 admin admin] PATH_INFO: `/v3/projects/50baf84541494e7596c3d73a3214cb95` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:40.823830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a422ec92-4c30-496b-bceb-04f3849373e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50baf84541494e7596c3d73a3214cb95)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:40.827977 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a422ec92-4c30-496b-bceb-04f3849373e8 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:40.851591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c468738f-2577-4ba8-a749-91eecfa478c6 admin admin] The token cache is being invalidate because project b9c72b43e6aa4911a1b441e47c193fab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:12:40.856463 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c468738f-2577-4ba8-a749-91eecfa478c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:40.857686 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c468738f-2577-4ba8-a749-91eecfa478c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:40.863625 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 820/1643] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:12:40 2022] DELETE /identity/v3/projects/b9c72b43e6aa4911a1b441e47c193fab => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:40.863853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:40.865302 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468738f-2577-4ba8-a749-91eecfa478c6 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:40.870426 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c468738f-2577-4ba8-a749-91eecfa478c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:40.907700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468738f-2577-4ba8-a749-91eecfa478c6 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:40.935489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-a422ec92-4c30-496b-bceb-04f3849373e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50baf84541494e7596c3d73a3214cb95', 'request_id': 'req-a422ec92-4c30-496b-bceb-04f3849373e8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:40.937982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:40.953081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-a422ec92-4c30-496b-bceb-04f3849373e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50baf84541494e7596c3d73a3214cb95', 'request_id': 'req-a422ec92-4c30-496b-bceb-04f3849373e8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:40.973985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-a422ec92-4c30-496b-bceb-04f3849373e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50baf84541494e7596c3d73a3214cb95', 'request_id': 'req-a422ec92-4c30-496b-bceb-04f3849373e8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:41.000167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-a422ec92-4c30-496b-bceb-04f3849373e8 admin admin] The token cache is being invalidate because project 50baf84541494e7596c3d73a3214cb95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:12:41.003503 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-a422ec92-4c30-496b-bceb-04f3849373e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:41.004494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-a422ec92-4c30-496b-bceb-04f3849373e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:41.007050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 824/1644] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:12:40 2022] DELETE /identity/v3/projects/50baf84541494e7596c3d73a3214cb95 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:12:41.008491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a422ec92-4c30-496b-bceb-04f3849373e8 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:41.011148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a422ec92-4c30-496b-bceb-04f3849373e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:41.053837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe79e2-9867-47e2-a4d8-33d22dacf570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:41.054780 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe79e2-9867-47e2-a4d8-33d22dacf570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:41.055199 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe79e2-9867-47e2-a4d8-33d22dacf570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:41.055840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe79e2-9867-47e2-a4d8-33d22dacf570 admin admin] PATH_INFO: `/v3/users/307f7fbfe4ff45daba698c930b1faff3` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:41.058014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe79e2-9867-47e2-a4d8-33d22dacf570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=307f7fbfe4ff45daba698c930b1faff3)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:41.062498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe79e2-9867-47e2-a4d8-33d22dacf570 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:41.173220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-47fe79e2-9867-47e2-a4d8-33d22dacf570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '307f7fbfe4ff45daba698c930b1faff3'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:41.208836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-47fe79e2-9867-47e2-a4d8-33d22dacf570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '307f7fbfe4ff45daba698c930b1faff3'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:41.244565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-47fe79e2-9867-47e2-a4d8-33d22dacf570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '307f7fbfe4ff45daba698c930b1faff3'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:41.251226 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-47fe79e2-9867-47e2-a4d8-33d22dacf570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '307f7fbfe4ff45daba698c930b1faff3'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:41.282672 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-47fe79e2-9867-47e2-a4d8-33d22dacf570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '307f7fbfe4ff45daba698c930b1faff3'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:41.292942 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a422ec92-4c30-496b-bceb-04f3849373e8 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:41.295077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 821/1645] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:12:40 2022] DELETE /identity/v3/users/307f7fbfe4ff45daba698c930b1faff3 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:41.296553 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c4c00-34ce-41d9-9b67-7a8a853aac24 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:41.296855 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c4c00-34ce-41d9-9b67-7a8a853aac24 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:41.297139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c4c00-34ce-41d9-9b67-7a8a853aac24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:41.353401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-884c4c00-34ce-41d9-9b67-7a8a853aac24 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:41.357744 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-884c4c00-34ce-41d9-9b67-7a8a853aac24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:41.381049 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18caee9-586c-4444-a777-209b64c71574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:41.382161 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18caee9-586c-4444-a777-209b64c71574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:41.382847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18caee9-586c-4444-a777-209b64c71574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:41.383802 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18caee9-586c-4444-a777-209b64c71574 admin admin] PATH_INFO: `/v3/users/aeb96f2b075140ab917d005955d0156f` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:41.385612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18caee9-586c-4444-a777-209b64c71574 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeb96f2b075140ab917d005955d0156f)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:41.391971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18caee9-586c-4444-a777-209b64c71574 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:41.492403 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-c18caee9-586c-4444-a777-209b64c71574 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb96f2b075140ab917d005955d0156f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:41.509931 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:41.520559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 822/1646] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:41.522010 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e2bba-2ee1-4c52-9c2d-7c3379970f75 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:41.522306 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e2bba-2ee1-4c52-9c2d-7c3379970f75 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:41.522585 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e2bba-2ee1-4c52-9c2d-7c3379970f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:41.533617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-c18caee9-586c-4444-a777-209b64c71574 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb96f2b075140ab917d005955d0156f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:41.556549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-c18caee9-586c-4444-a777-209b64c71574 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb96f2b075140ab917d005955d0156f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:41.557484 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-c18caee9-586c-4444-a777-209b64c71574 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb96f2b075140ab917d005955d0156f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:41.565011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-214e2bba-2ee1-4c52-9c2d-7c3379970f75 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:41.566317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-214e2bba-2ee1-4c52-9c2d-7c3379970f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:41.569070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-c18caee9-586c-4444-a777-209b64c71574 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb96f2b075140ab917d005955d0156f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:41.586193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 825/1647] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:12:41 2022] DELETE /identity/v3/users/aeb96f2b075140ab917d005955d0156f => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:41.589105 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a4faa-7989-42df-b2cc-5d471a1cfbb7 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:41.589715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a4faa-7989-42df-b2cc-5d471a1cfbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:41.590311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 823/1648] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:41.590799 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a4faa-7989-42df-b2cc-5d471a1cfbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:41.611021 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-025a4faa-7989-42df-b2cc-5d471a1cfbb7 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:41.612524 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-025a4faa-7989-42df-b2cc-5d471a1cfbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:41.644452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e2bba-2ee1-4c52-9c2d-7c3379970f75 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:41.646467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-214e2bba-2ee1-4c52-9c2d-7c3379970f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:41.719486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:41.733421 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 826/1649] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:41.738346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a4faa-7989-42df-b2cc-5d471a1cfbb7 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:41.802400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a4faa-7989-42df-b2cc-5d471a1cfbb7 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:41.820026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d685437-66da-409d-87fe-364cdb346c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:41.823881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d685437-66da-409d-87fe-364cdb346c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:41.824486 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d685437-66da-409d-87fe-364cdb346c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:41.824952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d685437-66da-409d-87fe-364cdb346c7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:41.826047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d685437-66da-409d-87fe-364cdb346c7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:12:41.827600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d685437-66da-409d-87fe-364cdb346c7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:41.832375 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d685437-66da-409d-87fe-364cdb346c7b admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:41.837864 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e2bba-2ee1-4c52-9c2d-7c3379970f75 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:41.895043 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 827/1650] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:12:41.899610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d685437-66da-409d-87fe-364cdb346c7b admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:41.930287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d685437-66da-409d-87fe-364cdb346c7b admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:41.972627 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e26847-5ce7-4a35-81ce-0cf5dc72cd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:41.976584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e26847-5ce7-4a35-81ce-0cf5dc72cd14 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:41.976584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e26847-5ce7-4a35-81ce-0cf5dc72cd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:41.976584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e26847-5ce7-4a35-81ce-0cf5dc72cd14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:41.979428 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e26847-5ce7-4a35-81ce-0cf5dc72cd14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:41.987331 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e26847-5ce7-4a35-81ce-0cf5dc72cd14 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:42.059305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7e808-d885-454c-836f-5b4faed5063a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:42.060848 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7e808-d885-454c-836f-5b4faed5063a service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:42.067072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7e808-d885-454c-836f-5b4faed5063a service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:42.068342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7e808-d885-454c-836f-5b4faed5063a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:42.074319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 828/1651] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:41 2022] POST /identity/v3/projects => generated 371 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:42.086102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e26847-5ce7-4a35-81ce-0cf5dc72cd14 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:42.106100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7e808-d885-454c-836f-5b4faed5063a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:42.108194 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e7e808-d885-454c-836f-5b4faed5063a service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:42.144113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 824/1652] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:42.198451 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e26847-5ce7-4a35-81ce-0cf5dc72cd14 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:42.203916 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e05ed7-922a-475c-a6d1-a915737ff23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:42.204775 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e05ed7-922a-475c-a6d1-a915737ff23b admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:42.205441 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e05ed7-922a-475c-a6d1-a915737ff23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:42.205441 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e05ed7-922a-475c-a6d1-a915737ff23b admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:42.206033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e05ed7-922a-475c-a6d1-a915737ff23b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:42.208273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e05ed7-922a-475c-a6d1-a915737ff23b admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:42.250822 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 829/1653] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:42 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:42.254703 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7e808-d885-454c-836f-5b4faed5063a service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:42.270042 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7e808-d885-454c-836f-5b4faed5063a service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:42.279287 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f233af-eaf8-4143-ae71-f989360aa0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:42.281731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f233af-eaf8-4143-ae71-f989360aa0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:42.282084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f233af-eaf8-4143-ae71-f989360aa0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:42.282366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f233af-eaf8-4143-ae71-f989360aa0a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:42.284690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f233af-eaf8-4143-ae71-f989360aa0a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:42.289976 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f233af-eaf8-4143-ae71-f989360aa0a0 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:42.332745 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e05ed7-922a-475c-a6d1-a915737ff23b admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:42.334599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-93e05ed7-922a-475c-a6d1-a915737ff23b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:42.338817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 825/1654] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:42.339590 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f233af-eaf8-4143-ae71-f989360aa0a0 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:42.373093 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e05ed7-922a-475c-a6d1-a915737ff23b admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:42.396154 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f233af-eaf8-4143-ae71-f989360aa0a0 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:42.401425 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeba3d3-150f-4bea-947f-474ef1f111de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:42.407554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeba3d3-150f-4bea-947f-474ef1f111de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:42.413152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeba3d3-150f-4bea-947f-474ef1f111de admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:42.413790 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeba3d3-150f-4bea-947f-474ef1f111de admin admin] PATH_INFO: `/v3/projects/360fdbda7e244170b6c0d1c183af3b6c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:42.418474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeba3d3-150f-4bea-947f-474ef1f111de admin admin] RBAC: Authorizing `identity:delete_project(project_id=360fdbda7e244170b6c0d1c183af3b6c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:42.420876 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed09938-cb56-41ba-ac9b-bd5cc409ca8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:42.421763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed09938-cb56-41ba-ac9b-bd5cc409ca8a admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:42.422060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed09938-cb56-41ba-ac9b-bd5cc409ca8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:42.422394 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed09938-cb56-41ba-ac9b-bd5cc409ca8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:42.423849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed09938-cb56-41ba-ac9b-bd5cc409ca8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:42.427605 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed09938-cb56-41ba-ac9b-bd5cc409ca8a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:42.432324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeba3d3-150f-4bea-947f-474ef1f111de admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:42.502558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 826/1655] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:42.505840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed09938-cb56-41ba-ac9b-bd5cc409ca8a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:42.546493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed09938-cb56-41ba-ac9b-bd5cc409ca8a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:42.555732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:42.558121 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ea287-f74a-4bb8-a0d9-53f2a17fe7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:42.565626 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ea287-f74a-4bb8-a0d9-53f2a17fe7ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:42.566099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ea287-f74a-4bb8-a0d9-53f2a17fe7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:42.566688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ea287-f74a-4bb8-a0d9-53f2a17fe7ab admin admin] PATH_INFO: `/v3/projects/8276462b52db454c8a859cba0096d475/users/6895a002562c4cf0b13ddfe4e5dbf26d/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:42.599249 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ea287-f74a-4bb8-a0d9-53f2a17fe7ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8276462b52db454c8a859cba0096d475, user_id=6895a002562c4cf0b13ddfe4e5dbf26d, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:42.602389 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ea287-f74a-4bb8-a0d9-53f2a17fe7ab admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:42.638784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 827/1656] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:42 2022] PUT /identity/v3/projects/8276462b52db454c8a859cba0096d475/users/6895a002562c4cf0b13ddfe4e5dbf26d/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:42.654388 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f914c49-88f9-44ea-90a4-98748b107694 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:42.655614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f914c49-88f9-44ea-90a4-98748b107694 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:42.655924 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f914c49-88f9-44ea-90a4-98748b107694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:42.674644 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-eeeba3d3-150f-4bea-947f-474ef1f111de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '360fdbda7e244170b6c0d1c183af3b6c', 'request_id': 'req-eeeba3d3-150f-4bea-947f-474ef1f111de'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:42.732613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-eeeba3d3-150f-4bea-947f-474ef1f111de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '360fdbda7e244170b6c0d1c183af3b6c', 'request_id': 'req-eeeba3d3-150f-4bea-947f-474ef1f111de'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:42.739160 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-6f914c49-88f9-44ea-90a4-98748b107694 None None] MFA Rules not processed for user `6895a002562c4cf0b13ddfe4e5dbf26d`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:42.742397 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6f914c49-88f9-44ea-90a4-98748b107694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:42.763093 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-eeeba3d3-150f-4bea-947f-474ef1f111de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '360fdbda7e244170b6c0d1c183af3b6c', 'request_id': 'req-eeeba3d3-150f-4bea-947f-474ef1f111de'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:42.833967 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-eeeba3d3-150f-4bea-947f-474ef1f111de admin admin] The token cache is being invalidate because project 360fdbda7e244170b6c0d1c183af3b6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:12:42.844724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-eeeba3d3-150f-4bea-947f-474ef1f111de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:42.846012 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-eeeba3d3-150f-4bea-947f-474ef1f111de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:42.866509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 830/1657] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:12:42 2022] DELETE /identity/v3/projects/360fdbda7e244170b6c0d1c183af3b6c => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:42.872661 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e020e7c-2025-454d-a663-4056b2970dce None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:42.873159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e020e7c-2025-454d-a663-4056b2970dce None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:42.876495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e020e7c-2025-454d-a663-4056b2970dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:43.049811 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-6e020e7c-2025-454d-a663-4056b2970dce None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:43.062769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6e020e7c-2025-454d-a663-4056b2970dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:43.252302 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 828/1658] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 599 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:43.261692 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f914c49-88f9-44ea-90a4-98748b107694 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:43.266154 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6f914c49-88f9-44ea-90a4-98748b107694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:43.320337 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 831/1659] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:43.346122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb1813-719a-47d7-8724-2a9a4d389fd2 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:43.346565 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb1813-719a-47d7-8724-2a9a4d389fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:43.347095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb1813-719a-47d7-8724-2a9a4d389fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:43.404639 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:43.405040 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-c6fb1813-719a-47d7-8724-2a9a4d389fd2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:43.406568 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c6fb1813-719a-47d7-8724-2a9a4d389fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:43.424072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 832/1660] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:43.425365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb1813-719a-47d7-8724-2a9a4d389fd2 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:43.444860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb1813-719a-47d7-8724-2a9a4d389fd2 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:43.473587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a22b7-d976-4dec-b6fe-aade58948596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:43.474550 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a22b7-d976-4dec-b6fe-aade58948596 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:43.475157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a22b7-d976-4dec-b6fe-aade58948596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:43.475631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a22b7-d976-4dec-b6fe-aade58948596 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:43.476705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a22b7-d976-4dec-b6fe-aade58948596 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:12:43.477191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a22b7-d976-4dec-b6fe-aade58948596 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:43.482130 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a22b7-d976-4dec-b6fe-aade58948596 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:43.524272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 833/1661] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:12:43.525519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a22b7-d976-4dec-b6fe-aade58948596 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:43.543752 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f914c49-88f9-44ea-90a4-98748b107694 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:43.549209 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a22b7-d976-4dec-b6fe-aade58948596 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:43.555824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81df0e-dd22-4579-9174-25c417ea321d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:43.556924 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81df0e-dd22-4579-9174-25c417ea321d admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:43.559252 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81df0e-dd22-4579-9174-25c417ea321d admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:43.560770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81df0e-dd22-4579-9174-25c417ea321d admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:43.561689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81df0e-dd22-4579-9174-25c417ea321d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:43.579368 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15271252-87ba-46fb-ae73-7307746b477e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:43.580413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15271252-87ba-46fb-ae73-7307746b477e admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:43.581284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f81df0e-dd22-4579-9174-25c417ea321d admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:43.581742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15271252-87ba-46fb-ae73-7307746b477e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:43.582215 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15271252-87ba-46fb-ae73-7307746b477e admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:43.583489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15271252-87ba-46fb-ae73-7307746b477e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:43.587476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15271252-87ba-46fb-ae73-7307746b477e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:43.665268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 834/1662] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:43 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:43.667138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15271252-87ba-46fb-ae73-7307746b477e admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:43.672128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 829/1663] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:43 2022] POST /identity/v3/projects => generated 373 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:43.676931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81df0e-dd22-4579-9174-25c417ea321d admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:43.855282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81df0e-dd22-4579-9174-25c417ea321d admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:43.873228 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd8bab-35dd-40c6-9a6b-9f231fa832b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:43.873941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd8bab-35dd-40c6-9a6b-9f231fa832b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:43.874228 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd8bab-35dd-40c6-9a6b-9f231fa832b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:43.874510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd8bab-35dd-40c6-9a6b-9f231fa832b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:43.875472 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd8bab-35dd-40c6-9a6b-9f231fa832b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:43.877172 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd8bab-35dd-40c6-9a6b-9f231fa832b7 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:43.896956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15271252-87ba-46fb-ae73-7307746b477e admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:43.917532 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5de65e-f9c5-427a-8e3e-a13c4b60fad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:43.919681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5de65e-f9c5-427a-8e3e-a13c4b60fad6 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:43.919965 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5de65e-f9c5-427a-8e3e-a13c4b60fad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:43.924788 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5de65e-f9c5-427a-8e3e-a13c4b60fad6 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:43.925992 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5de65e-f9c5-427a-8e3e-a13c4b60fad6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:43.926336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 830/1664] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:43 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:43.928122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5de65e-f9c5-427a-8e3e-a13c4b60fad6 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:43.930070 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd8bab-35dd-40c6-9a6b-9f231fa832b7 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:43.960812 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd8bab-35dd-40c6-9a6b-9f231fa832b7 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:43.963835 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:43.975987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 835/1665] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:43 2022] POST /identity/v3/users => generated 287 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:43.977342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5de65e-f9c5-427a-8e3e-a13c4b60fad6 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:43.980004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cba6b-57f9-416d-b105-155e7fd45ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:43.994602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5de65e-f9c5-427a-8e3e-a13c4b60fad6 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:43.995632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cba6b-57f9-416d-b105-155e7fd45ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:43.996179 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cba6b-57f9-416d-b105-155e7fd45ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:43.996589 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cba6b-57f9-416d-b105-155e7fd45ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:43.997670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29cba6b-57f9-416d-b105-155e7fd45ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:43.999561 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29cba6b-57f9-416d-b105-155e7fd45ac3 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:44.003292 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c9d5d-d7f8-43e9-83b0-51da3a6b7c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:44.004515 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c9d5d-d7f8-43e9-83b0-51da3a6b7c51 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:44.005198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c9d5d-d7f8-43e9-83b0-51da3a6b7c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:44.005614 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c9d5d-d7f8-43e9-83b0-51da3a6b7c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:44.007233 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c9d5d-d7f8-43e9-83b0-51da3a6b7c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:44.010960 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c9d5d-d7f8-43e9-83b0-51da3a6b7c51 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:44.026921 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 836/1666] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:44.028158 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c9d5d-d7f8-43e9-83b0-51da3a6b7c51 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:44.030443 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 831/1667] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:44.034381 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cba6b-57f9-416d-b105-155e7fd45ac3 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:44.039953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c9d5d-d7f8-43e9-83b0-51da3a6b7c51 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:44.046928 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de761ce-83e7-4500-9907-33e07f7e9172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:44.047966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cba6b-57f9-416d-b105-155e7fd45ac3 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:44.048149 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de761ce-83e7-4500-9907-33e07f7e9172 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:44.048559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de761ce-83e7-4500-9907-33e07f7e9172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:44.048970 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de761ce-83e7-4500-9907-33e07f7e9172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:44.049835 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de761ce-83e7-4500-9907-33e07f7e9172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:44.051617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de761ce-83e7-4500-9907-33e07f7e9172 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:44.064497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3691f-89b4-40c1-9869-c50b1835de9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:44.068307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3691f-89b4-40c1-9869-c50b1835de9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:44.068307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3691f-89b4-40c1-9869-c50b1835de9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:44.068307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3691f-89b4-40c1-9869-c50b1835de9c admin admin] PATH_INFO: `/v3/projects/ad6278c315994028a3fc637d826a291d/users/80457bae80ab410cbef5d1e988270046/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:44.068565 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 837/1668] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:44.069596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de761ce-83e7-4500-9907-33e07f7e9172 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:44.099618 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de761ce-83e7-4500-9907-33e07f7e9172 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:44.109811 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3691f-89b4-40c1-9869-c50b1835de9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6278c315994028a3fc637d826a291d, user_id=80457bae80ab410cbef5d1e988270046, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:44.112371 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6724430-e37d-41f7-8677-1312be75fbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:44.113287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6724430-e37d-41f7-8677-1312be75fbb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:44.113570 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6724430-e37d-41f7-8677-1312be75fbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:44.113851 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6724430-e37d-41f7-8677-1312be75fbb6 admin admin] PATH_INFO: `/v3/projects/fd4be66441244536beb725008c0c39d1/users/c94ba0f2814c41a59f000e80ba423cb2/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:44.121074 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3691f-89b4-40c1-9869-c50b1835de9c admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:44.157385 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6724430-e37d-41f7-8677-1312be75fbb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd4be66441244536beb725008c0c39d1, user_id=c94ba0f2814c41a59f000e80ba423cb2, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:44.162046 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 832/1669] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:44 2022] PUT /identity/v3/projects/ad6278c315994028a3fc637d826a291d/users/80457bae80ab410cbef5d1e988270046/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:44.162321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6724430-e37d-41f7-8677-1312be75fbb6 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:44.164527 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3691f-89b4-40c1-9869-c50b1835de9c admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:44.212815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 838/1670] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:44 2022] PUT /identity/v3/projects/fd4be66441244536beb725008c0c39d1/users/c94ba0f2814c41a59f000e80ba423cb2/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:44.214953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0da64f-1892-4fd2-b85a-656f036c5682 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:44.215254 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0da64f-1892-4fd2-b85a-656f036c5682 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:44.215541 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0da64f-1892-4fd2-b85a-656f036c5682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:44.313543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-7a0da64f-1892-4fd2-b85a-656f036c5682 None None] MFA Rules not processed for user `c94ba0f2814c41a59f000e80ba423cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:44.315214 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-7a0da64f-1892-4fd2-b85a-656f036c5682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:44.393230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3691f-89b4-40c1-9869-c50b1835de9c admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:44.437557 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:44.562719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 839/1671] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:44.567408 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0da64f-1892-4fd2-b85a-656f036c5682 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:44.615748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b8875-40c0-4e36-99e3-47cb82f81a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:44.616736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b8875-40c0-4e36-99e3-47cb82f81a00 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:44.617135 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b8875-40c0-4e36-99e3-47cb82f81a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:44.624342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b8875-40c0-4e36-99e3-47cb82f81a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:44.633104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b8875-40c0-4e36-99e3-47cb82f81a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:44.638513 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b8875-40c0-4e36-99e3-47cb82f81a00 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:44.662752 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 833/1672] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 499 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:44.668731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b8875-40c0-4e36-99e3-47cb82f81a00 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:44.700098 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b8875-40c0-4e36-99e3-47cb82f81a00 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:44.715339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37c87a-852f-423a-9b6f-33b9794869d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:44.717030 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37c87a-852f-423a-9b6f-33b9794869d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:44.717431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37c87a-852f-423a-9b6f-33b9794869d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:44.717983 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37c87a-852f-423a-9b6f-33b9794869d9 admin admin] PATH_INFO: `/v3/projects/ad6278c315994028a3fc637d826a291d/users/80457bae80ab410cbef5d1e988270046/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:44.735410 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37c87a-852f-423a-9b6f-33b9794869d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6278c315994028a3fc637d826a291d, user_id=80457bae80ab410cbef5d1e988270046, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:44.740439 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37c87a-852f-423a-9b6f-33b9794869d9 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:44.790202 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 834/1673] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:44 2022] PUT /identity/v3/projects/ad6278c315994028a3fc637d826a291d/users/80457bae80ab410cbef5d1e988270046/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:44.798140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d474b0a-0c39-4336-bc61-a7d468a182cf None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:44.798782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d474b0a-0c39-4336-bc61-a7d468a182cf None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:44.799339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d474b0a-0c39-4336-bc61-a7d468a182cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:44.811992 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0da64f-1892-4fd2-b85a-656f036c5682 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:44.892906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-2d474b0a-0c39-4336-bc61-a7d468a182cf None None] MFA Rules not processed for user `80457bae80ab410cbef5d1e988270046`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:44.897272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2d474b0a-0c39-4336-bc61-a7d468a182cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:45.000351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:45.051955 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9dbd8a-ef8d-4ee9-be1e-3061b98b47b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:45.052958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9dbd8a-ef8d-4ee9-be1e-3061b98b47b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:45.053606 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9dbd8a-ef8d-4ee9-be1e-3061b98b47b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:45.053606 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9dbd8a-ef8d-4ee9-be1e-3061b98b47b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:45.065027 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9dbd8a-ef8d-4ee9-be1e-3061b98b47b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:45.070477 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9dbd8a-ef8d-4ee9-be1e-3061b98b47b3 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:45.081603 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 835/1674] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:45.082627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d474b0a-0c39-4336-bc61-a7d468a182cf None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:45.086741 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2d474b0a-0c39-4336-bc61-a7d468a182cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:45.118475 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d474b0a-0c39-4336-bc61-a7d468a182cf None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:45.131453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5aeab-a921-493f-b294-3bf2af1a0f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:45.132647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5aeab-a921-493f-b294-3bf2af1a0f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:45.133103 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5aeab-a921-493f-b294-3bf2af1a0f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:45.133509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d5aeab-a921-493f-b294-3bf2af1a0f7b admin admin] PATH_INFO: `/v3/projects/f571b8b6410349418219f9d80d2df4fd` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:45.139783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5aeab-a921-493f-b294-3bf2af1a0f7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f571b8b6410349418219f9d80d2df4fd)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:45.141988 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d5aeab-a921-493f-b294-3bf2af1a0f7b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:45.174785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 840/1675] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:44 2022] POST /identity/v3/projects => generated 373 bytes in 608 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:45.177079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9dbd8a-ef8d-4ee9-be1e-3061b98b47b3 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:45.193801 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-7d9dbd8a-ef8d-4ee9-be1e-3061b98b47b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:45.256871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-87d5aeab-a921-493f-b294-3bf2af1a0f7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f571b8b6410349418219f9d80d2df4fd', 'request_id': 'req-87d5aeab-a921-493f-b294-3bf2af1a0f7b'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:45.284280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-87d5aeab-a921-493f-b294-3bf2af1a0f7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f571b8b6410349418219f9d80d2df4fd', 'request_id': 'req-87d5aeab-a921-493f-b294-3bf2af1a0f7b'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:45.300034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-87d5aeab-a921-493f-b294-3bf2af1a0f7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f571b8b6410349418219f9d80d2df4fd', 'request_id': 'req-87d5aeab-a921-493f-b294-3bf2af1a0f7b'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:45.325530 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-87d5aeab-a921-493f-b294-3bf2af1a0f7b admin admin] The token cache is being invalidate because project f571b8b6410349418219f9d80d2df4fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:12:45.326609 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-87d5aeab-a921-493f-b294-3bf2af1a0f7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:45.328368 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-87d5aeab-a921-493f-b294-3bf2af1a0f7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:45.337114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 836/1676] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:12:45 2022] DELETE /identity/v3/projects/f571b8b6410349418219f9d80d2df4fd => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:45.341617 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5aeab-a921-493f-b294-3bf2af1a0f7b admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:45.343724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-87d5aeab-a921-493f-b294-3bf2af1a0f7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:45.364725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:45.411006 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9dbd8a-ef8d-4ee9-be1e-3061b98b47b3 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:45.542084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d5aeab-a921-493f-b294-3bf2af1a0f7b admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:45.557395 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2888733-a1fc-4fa5-9d13-8188debaa451 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:45.558536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0ab29-2780-413f-8764-09f862ae24e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:45.559373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0ab29-2780-413f-8764-09f862ae24e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:45.560124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2888733-a1fc-4fa5-9d13-8188debaa451 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:45.560522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0ab29-2780-413f-8764-09f862ae24e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:45.560910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2888733-a1fc-4fa5-9d13-8188debaa451 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:45.561297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0ab29-2780-413f-8764-09f862ae24e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:45.561678 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2888733-a1fc-4fa5-9d13-8188debaa451 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:45.562497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0ab29-2780-413f-8764-09f862ae24e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:45.565602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0ab29-2780-413f-8764-09f862ae24e4 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:45.566822 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a2888733-a1fc-4fa5-9d13-8188debaa451 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:45.612813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 837/1677] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:45 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:45.614408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711a16f-5792-4f55-9ddf-ddf444f0d0ef None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:45.617591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711a16f-5792-4f55-9ddf-ddf444f0d0ef None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:45.618145 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711a16f-5792-4f55-9ddf-ddf444f0d0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:45.656355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-4711a16f-5792-4f55-9ddf-ddf444f0d0ef None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:45.662733 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-4711a16f-5792-4f55-9ddf-ddf444f0d0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:45.692096 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 838/1678] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:45.693502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711a16f-5792-4f55-9ddf-ddf444f0d0ef None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:45.729943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711a16f-5792-4f55-9ddf-ddf444f0d0ef None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:45.740730 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2888733-a1fc-4fa5-9d13-8188debaa451 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:45.746716 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba9f61-c922-4df9-a2d0-72b8e9c305a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:45.747859 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2888733-a1fc-4fa5-9d13-8188debaa451 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:45.751706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba9f61-c922-4df9-a2d0-72b8e9c305a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:45.752205 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba9f61-c922-4df9-a2d0-72b8e9c305a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:45.752630 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba9f61-c922-4df9-a2d0-72b8e9c305a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:45.753663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba9f61-c922-4df9-a2d0-72b8e9c305a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:45.756379 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba9f61-c922-4df9-a2d0-72b8e9c305a8 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:45.774034 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 841/1679] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:45 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:45.776948 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90a873-85ef-4051-8111-b31e254fbc89 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:45.778963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90a873-85ef-4051-8111-b31e254fbc89 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:45.779373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90a873-85ef-4051-8111-b31e254fbc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:45.786330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 839/1680] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:45.788024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba9f61-c922-4df9-a2d0-72b8e9c305a8 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:45.811260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba9f61-c922-4df9-a2d0-72b8e9c305a8 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:45.821414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc6d4d-5847-4b25-8b37-62973a919e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:45.823777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc6d4d-5847-4b25-8b37-62973a919e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:45.824999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-9b90a873-85ef-4051-8111-b31e254fbc89 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:45.825426 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc6d4d-5847-4b25-8b37-62973a919e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:45.826893 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc6d4d-5847-4b25-8b37-62973a919e74 admin admin] PATH_INFO: `/v3/projects/9bcbe192603343a89489976323eb5205/users/f4ef3cbbbe624be3a8b294732b0bf90a/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:45.828106 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-9b90a873-85ef-4051-8111-b31e254fbc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:45.854277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 842/1681] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:45.855684 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90a873-85ef-4051-8111-b31e254fbc89 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:45.867338 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc6d4d-5847-4b25-8b37-62973a919e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bcbe192603343a89489976323eb5205, user_id=f4ef3cbbbe624be3a8b294732b0bf90a, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:45.872368 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc6d4d-5847-4b25-8b37-62973a919e74 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:45.878976 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90a873-85ef-4051-8111-b31e254fbc89 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:45.887290 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171421ad-771a-417b-8562-7d1786b2806c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:45.888193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171421ad-771a-417b-8562-7d1786b2806c admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:45.888887 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171421ad-771a-417b-8562-7d1786b2806c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:45.889322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171421ad-771a-417b-8562-7d1786b2806c admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:45.897762 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171421ad-771a-417b-8562-7d1786b2806c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:12:45.898316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171421ad-771a-417b-8562-7d1786b2806c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:45.901599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171421ad-771a-417b-8562-7d1786b2806c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:45.905906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 840/1682] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:45 2022] PUT /identity/v3/projects/9bcbe192603343a89489976323eb5205/users/f4ef3cbbbe624be3a8b294732b0bf90a/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:45.907068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc6d4d-5847-4b25-8b37-62973a919e74 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:45.930932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 843/1683] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:12:45.937387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171421ad-771a-417b-8562-7d1786b2806c admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:45.965181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171421ad-771a-417b-8562-7d1786b2806c admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:46.003077 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:46.071989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc6d4d-5847-4b25-8b37-62973a919e74 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:46.080609 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b28f7b-81a2-42c5-9810-4cd09cbbec03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:46.081332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b28f7b-81a2-42c5-9810-4cd09cbbec03 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:46.081693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b28f7b-81a2-42c5-9810-4cd09cbbec03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:46.082049 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b28f7b-81a2-42c5-9810-4cd09cbbec03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:46.082905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b28f7b-81a2-42c5-9810-4cd09cbbec03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:46.084674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b28f7b-81a2-42c5-9810-4cd09cbbec03 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:46.103361 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 841/1684] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:46.107811 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b28f7b-81a2-42c5-9810-4cd09cbbec03 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:46.110963 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-87b28f7b-81a2-42c5-9810-4cd09cbbec03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:46.138620 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d7e9f-b7b3-4304-bb98-38a738ec367e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:46.140561 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d7e9f-b7b3-4304-bb98-38a738ec367e admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:46.140920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d7e9f-b7b3-4304-bb98-38a738ec367e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:46.141323 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d7e9f-b7b3-4304-bb98-38a738ec367e admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:46.142730 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d7e9f-b7b3-4304-bb98-38a738ec367e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:46.149020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d7e9f-b7b3-4304-bb98-38a738ec367e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:46.214217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 844/1685] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:45 2022] POST /identity/v3/projects => generated 379 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:46.215240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d7e9f-b7b3-4304-bb98-38a738ec367e admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:46.224335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b28f7b-81a2-42c5-9810-4cd09cbbec03 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:46.233697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:46.321694 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d81bf4-c6d0-4794-b206-0e4ef5d83534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:46.324083 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d81bf4-c6d0-4794-b206-0e4ef5d83534 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:46.324435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d81bf4-c6d0-4794-b206-0e4ef5d83534 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:46.325441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d81bf4-c6d0-4794-b206-0e4ef5d83534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:46.329157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f1d81bf4-c6d0-4794-b206-0e4ef5d83534 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:46.352058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d7e9f-b7b3-4304-bb98-38a738ec367e admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:46.374557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f84ddc-c180-48f1-b4ef-09177a898929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:46.375603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f84ddc-c180-48f1-b4ef-09177a898929 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:46.376773 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f84ddc-c180-48f1-b4ef-09177a898929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:46.377133 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f84ddc-c180-48f1-b4ef-09177a898929 admin admin] PATH_INFO: `/v3/projects/9bcbe192603343a89489976323eb5205/users/f4ef3cbbbe624be3a8b294732b0bf90a/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:46.385245 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f84ddc-c180-48f1-b4ef-09177a898929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bcbe192603343a89489976323eb5205, user_id=f4ef3cbbbe624be3a8b294732b0bf90a, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:46.388078 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f84ddc-c180-48f1-b4ef-09177a898929 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:46.426529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 845/1686] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:46 2022] PUT /identity/v3/projects/9bcbe192603343a89489976323eb5205/users/f4ef3cbbbe624be3a8b294732b0bf90a/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:46.441070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f84ddc-c180-48f1-b4ef-09177a898929 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:46.502634 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d81bf4-c6d0-4794-b206-0e4ef5d83534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:46.504128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d81bf4-c6d0-4794-b206-0e4ef5d83534 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:46.560101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 842/1687] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:46.561536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf56ed-e741-4bef-9593-58642dba8e4c None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:46.563299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf56ed-e741-4bef-9593-58642dba8e4c None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:46.563595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf56ed-e741-4bef-9593-58642dba8e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:46.614540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f84ddc-c180-48f1-b4ef-09177a898929 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:46.620782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-2dcf56ed-e741-4bef-9593-58642dba8e4c None None] MFA Rules not processed for user `f4ef3cbbbe624be3a8b294732b0bf90a`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:46.624645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2dcf56ed-e741-4bef-9593-58642dba8e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:46.625445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661e053-d0a2-4257-b244-09e474f858d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:46.627920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661e053-d0a2-4257-b244-09e474f858d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:46.628360 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661e053-d0a2-4257-b244-09e474f858d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:46.628801 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661e053-d0a2-4257-b244-09e474f858d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:46.630169 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661e053-d0a2-4257-b244-09e474f858d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:46.636467 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661e053-d0a2-4257-b244-09e474f858d2 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:46.695068 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 846/1688] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:46 2022] POST /identity/v3/users => generated 290 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:46.696106 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661e053-d0a2-4257-b244-09e474f858d2 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:46.727249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661e053-d0a2-4257-b244-09e474f858d2 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:46.753399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae86f67-df90-4221-8258-4898c76540e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:46.754107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae86f67-df90-4221-8258-4898c76540e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:46.754392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae86f67-df90-4221-8258-4898c76540e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:46.754670 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae86f67-df90-4221-8258-4898c76540e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:46.755443 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae86f67-df90-4221-8258-4898c76540e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:46.757096 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae86f67-df90-4221-8258-4898c76540e1 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:46.806923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 847/1689] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:46.808120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae86f67-df90-4221-8258-4898c76540e1 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:46.849103 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae86f67-df90-4221-8258-4898c76540e1 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:46.870658 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca4c7c-460c-45b9-b8dd-263fefa2a065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:46.873741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca4c7c-460c-45b9-b8dd-263fefa2a065 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:46.873970 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca4c7c-460c-45b9-b8dd-263fefa2a065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:46.874247 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca4c7c-460c-45b9-b8dd-263fefa2a065 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:46.879951 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca4c7c-460c-45b9-b8dd-263fefa2a065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:46.881616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca4c7c-460c-45b9-b8dd-263fefa2a065 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:46.897292 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 843/1690] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:46.914664 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf56ed-e741-4bef-9593-58642dba8e4c None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:46.924824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 848/1691] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:46.934903 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf56ed-e741-4bef-9593-58642dba8e4c None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:46.943275 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fbf52-e86b-47be-afdf-6082f1c273d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:46.944868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fbf52-e86b-47be-afdf-6082f1c273d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:46.945159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fbf52-e86b-47be-afdf-6082f1c273d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:46.945438 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fbf52-e86b-47be-afdf-6082f1c273d4 admin admin] PATH_INFO: `/v3/projects/167b09014aa3444b94694c2ce89eddab/users/a23ed2a3521a4a328e46eb914938660e/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:46.974786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca4c7c-460c-45b9-b8dd-263fefa2a065 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:46.976756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ccca4c7c-460c-45b9-b8dd-263fefa2a065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:46.996375 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:47.013216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fbf52-e86b-47be-afdf-6082f1c273d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=167b09014aa3444b94694c2ce89eddab, user_id=a23ed2a3521a4a328e46eb914938660e, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:47.031045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fbf52-e86b-47be-afdf-6082f1c273d4 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:47.092033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 844/1692] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:46 2022] PUT /identity/v3/projects/167b09014aa3444b94694c2ce89eddab/users/a23ed2a3521a4a328e46eb914938660e/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:47.097181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484c52d-974c-4085-aff8-b1679859f0ba None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:47.097601 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484c52d-974c-4085-aff8-b1679859f0ba None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:47.097897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484c52d-974c-4085-aff8-b1679859f0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:47.160555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-7484c52d-974c-4085-aff8-b1679859f0ba None None] MFA Rules not processed for user `a23ed2a3521a4a328e46eb914938660e`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:47.161801 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7484c52d-974c-4085-aff8-b1679859f0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:47.180557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca4c7c-460c-45b9-b8dd-263fefa2a065 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:47.191449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:47.295483 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf08a7-172a-4ee9-a16b-1a328ac158ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:47.296247 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf08a7-172a-4ee9-a16b-1a328ac158ed service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:47.296717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf08a7-172a-4ee9-a16b-1a328ac158ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:47.305266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf08a7-172a-4ee9-a16b-1a328ac158ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:47.309765 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 845/1693] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:47.314382 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484c52d-974c-4085-aff8-b1679859f0ba None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:47.445562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484c52d-974c-4085-aff8-b1679859f0ba None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:47.457515 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee26a2-c369-4862-9822-0ce3c23d4b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:47.458208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee26a2-c369-4862-9822-0ce3c23d4b9a admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:47.458493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee26a2-c369-4862-9822-0ce3c23d4b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:47.458799 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee26a2-c369-4862-9822-0ce3c23d4b9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:47.459042 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf08a7-172a-4ee9-a16b-1a328ac158ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:47.459513 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee26a2-c369-4862-9822-0ce3c23d4b9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:47.460809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf08a7-172a-4ee9-a16b-1a328ac158ed service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:47.461533 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee26a2-c369-4862-9822-0ce3c23d4b9a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:47.489750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 849/1694] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:12:47.524124 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 846/1695] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:47 2022] POST /identity/v3/projects => generated 381 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:47.525079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf08a7-172a-4ee9-a16b-1a328ac158ed service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:47.543818 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf08a7-172a-4ee9-a16b-1a328ac158ed service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:47.731098 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34ca2c-3909-466d-a727-e98f4472434f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:47.732105 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34ca2c-3909-466d-a727-e98f4472434f admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:47.732495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34ca2c-3909-466d-a727-e98f4472434f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:47.732874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34ca2c-3909-466d-a727-e98f4472434f admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:47.733674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34ca2c-3909-466d-a727-e98f4472434f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:47.735860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34ca2c-3909-466d-a727-e98f4472434f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:47.786231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 850/1696] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:47 2022] POST /identity/v3/users => generated 297 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:12:47.790190 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee26a2-c369-4862-9822-0ce3c23d4b9a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:47.805596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee26a2-c369-4862-9822-0ce3c23d4b9a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:47.831965 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53071c-b40c-4642-8a83-79a54f292ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:47.832668 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53071c-b40c-4642-8a83-79a54f292ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:47.832961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53071c-b40c-4642-8a83-79a54f292ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:47.833243 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53071c-b40c-4642-8a83-79a54f292ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:47.840518 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53071c-b40c-4642-8a83-79a54f292ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:47.845884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53071c-b40c-4642-8a83-79a54f292ff6 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:47.879709 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 847/1697] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:47.884260 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34ca2c-3909-466d-a727-e98f4472434f admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:47.889813 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:47.896268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34ca2c-3909-466d-a727-e98f4472434f admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:47.909778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb6bbe-9aeb-4a64-b8d9-7f69cb138fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:47.911009 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb6bbe-9aeb-4a64-b8d9-7f69cb138fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:47.911330 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb6bbe-9aeb-4a64-b8d9-7f69cb138fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:47.911665 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb6bbe-9aeb-4a64-b8d9-7f69cb138fbb admin admin] PATH_INFO: `/v3/projects/fcf422f003424bb1954e85f63280424a/users/6a9cbe19290842169632bda1d2b6a528/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:47.964546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb6bbe-9aeb-4a64-b8d9-7f69cb138fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcf422f003424bb1954e85f63280424a, user_id=6a9cbe19290842169632bda1d2b6a528, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:47.967298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb6bbe-9aeb-4a64-b8d9-7f69cb138fbb admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:48.019774 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 851/1698] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:47 2022] PUT /identity/v3/projects/fcf422f003424bb1954e85f63280424a/users/6a9cbe19290842169632bda1d2b6a528/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:48.020087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53071c-b40c-4642-8a83-79a54f292ff6 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:48.034289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53071c-b40c-4642-8a83-79a54f292ff6 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:48.168885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d32f4c5-182e-4b7f-9b92-ee6bc70d005f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:48.171556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32f4c5-182e-4b7f-9b92-ee6bc70d005f admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:48.171946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32f4c5-182e-4b7f-9b92-ee6bc70d005f admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:48.173140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32f4c5-182e-4b7f-9b92-ee6bc70d005f admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:48.175523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32f4c5-182e-4b7f-9b92-ee6bc70d005f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:48.180964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d32f4c5-182e-4b7f-9b92-ee6bc70d005f admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:48.216305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 848/1699] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:48.220409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb6bbe-9aeb-4a64-b8d9-7f69cb138fbb admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:48.238254 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb6bbe-9aeb-4a64-b8d9-7f69cb138fbb admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:48.243766 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52077f9-af35-461d-8214-a86477cade76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:48.246250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52077f9-af35-461d-8214-a86477cade76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:48.246710 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52077f9-af35-461d-8214-a86477cade76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:48.247133 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52077f9-af35-461d-8214-a86477cade76 admin admin] PATH_INFO: `/v3/projects/fcf422f003424bb1954e85f63280424a/users/6a9cbe19290842169632bda1d2b6a528/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:48.249724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52077f9-af35-461d-8214-a86477cade76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcf422f003424bb1954e85f63280424a, user_id=6a9cbe19290842169632bda1d2b6a528, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:48.253793 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52077f9-af35-461d-8214-a86477cade76 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:48.294598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 852/1700] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:48 2022] PUT /identity/v3/projects/fcf422f003424bb1954e85f63280424a/users/6a9cbe19290842169632bda1d2b6a528/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:12:48.306337 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5273b8-ee4d-4b5e-9228-938b8295bfa6 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:48.307958 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5273b8-ee4d-4b5e-9228-938b8295bfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:48.308655 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5273b8-ee4d-4b5e-9228-938b8295bfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:48.396031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-2a5273b8-ee4d-4b5e-9228-938b8295bfa6 None None] MFA Rules not processed for user `6a9cbe19290842169632bda1d2b6a528`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:48.397395 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2a5273b8-ee4d-4b5e-9228-938b8295bfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:48.427300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:48.434984 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52077f9-af35-461d-8214-a86477cade76 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:48.578321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 849/1701] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:48.592851 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5273b8-ee4d-4b5e-9228-938b8295bfa6 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:48.602581 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52077f9-af35-461d-8214-a86477cade76 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:48.620068 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f68a3-aa14-4360-95cd-872b53aa454b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:48.623244 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f68a3-aa14-4360-95cd-872b53aa454b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:48.624306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f68a3-aa14-4360-95cd-872b53aa454b admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:48.626454 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f68a3-aa14-4360-95cd-872b53aa454b admin admin] PATH_INFO: `/v3/users/c94ba0f2814c41a59f000e80ba423cb2` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:48.628047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f68a3-aa14-4360-95cd-872b53aa454b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c94ba0f2814c41a59f000e80ba423cb2)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:48.637011 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f68a3-aa14-4360-95cd-872b53aa454b admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:48.757696 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-183f68a3-aa14-4360-95cd-872b53aa454b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c94ba0f2814c41a59f000e80ba423cb2'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:48.782097 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-183f68a3-aa14-4360-95cd-872b53aa454b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c94ba0f2814c41a59f000e80ba423cb2'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:48.793789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5273b8-ee4d-4b5e-9228-938b8295bfa6 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:48.801899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-183f68a3-aa14-4360-95cd-872b53aa454b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c94ba0f2814c41a59f000e80ba423cb2'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:48.802464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-183f68a3-aa14-4360-95cd-872b53aa454b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c94ba0f2814c41a59f000e80ba423cb2'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:48.806288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd931c35-8cf4-4078-b3cf-f43477d9dbb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:48.806288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd931c35-8cf4-4078-b3cf-f43477d9dbb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:48.807088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd931c35-8cf4-4078-b3cf-f43477d9dbb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:48.807384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd931c35-8cf4-4078-b3cf-f43477d9dbb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:48.824748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-183f68a3-aa14-4360-95cd-872b53aa454b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c94ba0f2814c41a59f000e80ba423cb2'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:48.828461 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd931c35-8cf4-4078-b3cf-f43477d9dbb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:48.833918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 853/1702] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:12:48 2022] DELETE /identity/v3/users/c94ba0f2814c41a59f000e80ba423cb2 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:48.837000 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd931c35-8cf4-4078-b3cf-f43477d9dbb4 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:48.839570 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec1e6b-8961-4f67-bbc6-363a80513c20 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:48.839982 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec1e6b-8961-4f67-bbc6-363a80513c20 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:48.841932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec1e6b-8961-4f67-bbc6-363a80513c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:48.881293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 850/1703] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:12:48.898344 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a3ec1e6b-8961-4f67-bbc6-363a80513c20 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:48.905347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a3ec1e6b-8961-4f67-bbc6-363a80513c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:49.104889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:49.112352 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd931c35-8cf4-4078-b3cf-f43477d9dbb4 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:49.122978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd931c35-8cf4-4078-b3cf-f43477d9dbb4 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:49.224585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 854/1704] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:49.272640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec1e6b-8961-4f67-bbc6-363a80513c20 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:49.274642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a3ec1e6b-8961-4f67-bbc6-363a80513c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:49.379471 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480d686-0981-4004-9b55-fe620d99a648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:49.380286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480d686-0981-4004-9b55-fe620d99a648 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:49.380639 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480d686-0981-4004-9b55-fe620d99a648 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:49.380990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480d686-0981-4004-9b55-fe620d99a648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:49.385457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c480d686-0981-4004-9b55-fe620d99a648 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:49.493256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec1e6b-8961-4f67-bbc6-363a80513c20 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:49.502445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c9574-12af-4345-a663-61dc43ffb8ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:49.504151 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c9574-12af-4345-a663-61dc43ffb8ec service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:49.506008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c9574-12af-4345-a663-61dc43ffb8ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:49.506313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c9574-12af-4345-a663-61dc43ffb8ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:49.529608 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480d686-0981-4004-9b55-fe620d99a648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:49.531098 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480d686-0981-4004-9b55-fe620d99a648 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:49.536824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c9574-12af-4345-a663-61dc43ffb8ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:49.538277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c9574-12af-4345-a663-61dc43ffb8ec service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:49.555612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 851/1705] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:49.557329 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480d686-0981-4004-9b55-fe620d99a648 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:49.560272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 855/1706] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:12:49.563730 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c480d686-0981-4004-9b55-fe620d99a648 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:49.583850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:49.589869 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480d686-0981-4004-9b55-fe620d99a648 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:49.598958 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68ad43-75dc-4f1e-84e6-d7fcc318f1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:49.600074 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68ad43-75dc-4f1e-84e6-d7fcc318f1d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:49.600499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68ad43-75dc-4f1e-84e6-d7fcc318f1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:49.600913 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68ad43-75dc-4f1e-84e6-d7fcc318f1d1 admin admin] PATH_INFO: `/v3/users/43e7cbc8663c4d5bb4812dd818f744aa` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:49.602279 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68ad43-75dc-4f1e-84e6-d7fcc318f1d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43e7cbc8663c4d5bb4812dd818f744aa)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:49.608849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68ad43-75dc-4f1e-84e6-d7fcc318f1d1 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:49.719726 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5a68ad43-75dc-4f1e-84e6-d7fcc318f1d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43e7cbc8663c4d5bb4812dd818f744aa'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:49.767638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5a68ad43-75dc-4f1e-84e6-d7fcc318f1d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43e7cbc8663c4d5bb4812dd818f744aa'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:49.793470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5a68ad43-75dc-4f1e-84e6-d7fcc318f1d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43e7cbc8663c4d5bb4812dd818f744aa'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:49.795413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5a68ad43-75dc-4f1e-84e6-d7fcc318f1d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43e7cbc8663c4d5bb4812dd818f744aa'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:49.823690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5a68ad43-75dc-4f1e-84e6-d7fcc318f1d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43e7cbc8663c4d5bb4812dd818f744aa'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:49.836577 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 852/1707] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:12:49 2022] DELETE /identity/v3/users/43e7cbc8663c4d5bb4812dd818f744aa => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:49.841999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1a0f9-61fd-4273-971b-faaddc54a1ba None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:49.842531 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1a0f9-61fd-4273-971b-faaddc54a1ba None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:49.842973 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1a0f9-61fd-4273-971b-faaddc54a1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:49.895228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-e1b1a0f9-61fd-4273-971b-faaddc54a1ba None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:49.896851 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e1b1a0f9-61fd-4273-971b-faaddc54a1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:50.037250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 856/1708] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:12:50.102367 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68ad43-75dc-4f1e-84e6-d7fcc318f1d1 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:50.104882 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5a68ad43-75dc-4f1e-84e6-d7fcc318f1d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:50.243297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68ad43-75dc-4f1e-84e6-d7fcc318f1d1 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:50.249028 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590939f5-594d-4f45-a84a-b6f0936ddf3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:50.249911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590939f5-594d-4f45-a84a-b6f0936ddf3b service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:50.252717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590939f5-594d-4f45-a84a-b6f0936ddf3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:50.253458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590939f5-594d-4f45-a84a-b6f0936ddf3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:50.268614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590939f5-594d-4f45-a84a-b6f0936ddf3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:50.270429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590939f5-594d-4f45-a84a-b6f0936ddf3b service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:50.293836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 853/1709] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:51.052349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e4bdd-bdb7-40b7-89d8-f5e0f93c1fd5 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:51.052890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e4bdd-bdb7-40b7-89d8-f5e0f93c1fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:51.052890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e4bdd-bdb7-40b7-89d8-f5e0f93c1fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:51.143528 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-5a4e4bdd-bdb7-40b7-89d8-f5e0f93c1fd5 None None] MFA Rules not processed for user `abdb339c5f5740d6aff4dfd1066f9384`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:51.150965 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5a4e4bdd-bdb7-40b7-89d8-f5e0f93c1fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:51.347169 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 857/1710] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:51.358605 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1201ac3-19e4-4725-a723-95122e3b21e0 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:51.359455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1201ac3-19e4-4725-a723-95122e3b21e0 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:51.359974 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1201ac3-19e4-4725-a723-95122e3b21e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:51.417603 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-e1201ac3-19e4-4725-a723-95122e3b21e0 None None] MFA Rules not processed for user `abdb339c5f5740d6aff4dfd1066f9384`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:51.420376 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e1201ac3-19e4-4725-a723-95122e3b21e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:51.453401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 854/1711] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:51.470879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e4bdd-bdb7-40b7-89d8-f5e0f93c1fd5 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:51.472897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5a4e4bdd-bdb7-40b7-89d8-f5e0f93c1fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:51.544263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:51.720850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1201ac3-19e4-4725-a723-95122e3b21e0 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:51.722381 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e1201ac3-19e4-4725-a723-95122e3b21e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:51.750071 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e4bdd-bdb7-40b7-89d8-f5e0f93c1fd5 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:51.769715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25998152-acbf-410b-9be0-6cf5163bc6de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:51.772759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25998152-acbf-410b-9be0-6cf5163bc6de service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:51.775251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25998152-acbf-410b-9be0-6cf5163bc6de service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:51.776487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25998152-acbf-410b-9be0-6cf5163bc6de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:51.779527 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1201ac3-19e4-4725-a723-95122e3b21e0 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:51.797088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23ae69-ee6a-41dd-97f7-21a635c23c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:51.810825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23ae69-ee6a-41dd-97f7-21a635c23c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:51.813823 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23ae69-ee6a-41dd-97f7-21a635c23c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:51.819533 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25998152-acbf-410b-9be0-6cf5163bc6de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:51.819964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23ae69-ee6a-41dd-97f7-21a635c23c76 admin admin] PATH_INFO: `/v3/projects/fd4be66441244536beb725008c0c39d1` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:51.824236 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23ae69-ee6a-41dd-97f7-21a635c23c76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd4be66441244536beb725008c0c39d1)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:51.825044 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25998152-acbf-410b-9be0-6cf5163bc6de service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:51.832616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23ae69-ee6a-41dd-97f7-21a635c23c76 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:51.873238 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 858/1712] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:51.877522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25998152-acbf-410b-9be0-6cf5163bc6de service nova] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:51.885056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-25998152-acbf-410b-9be0-6cf5163bc6de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:51.947273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25998152-acbf-410b-9be0-6cf5163bc6de service nova] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:51.989253 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-9f23ae69-ee6a-41dd-97f7-21a635c23c76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd4be66441244536beb725008c0c39d1', 'request_id': 'req-9f23ae69-ee6a-41dd-97f7-21a635c23c76'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:51.998148 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-9f23ae69-ee6a-41dd-97f7-21a635c23c76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd4be66441244536beb725008c0c39d1', 'request_id': 'req-9f23ae69-ee6a-41dd-97f7-21a635c23c76'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:52.008166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-9f23ae69-ee6a-41dd-97f7-21a635c23c76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd4be66441244536beb725008c0c39d1', 'request_id': 'req-9f23ae69-ee6a-41dd-97f7-21a635c23c76'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:52.016575 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:52.050102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-9f23ae69-ee6a-41dd-97f7-21a635c23c76 admin admin] The token cache is being invalidate because project fd4be66441244536beb725008c0c39d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:12:52.050102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-9f23ae69-ee6a-41dd-97f7-21a635c23c76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:52.051612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-9f23ae69-ee6a-41dd-97f7-21a635c23c76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:52.065059 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 855/1713] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:12:51 2022] DELETE /identity/v3/projects/fd4be66441244536beb725008c0c39d1 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:52.076984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23ae69-ee6a-41dd-97f7-21a635c23c76 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:52.079932 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9f23ae69-ee6a-41dd-97f7-21a635c23c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:52.145982 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4c9de-8fb4-4fbf-9aa4-c1c816dfebf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:52.146897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4c9de-8fb4-4fbf-9aa4-c1c816dfebf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:52.147180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4c9de-8fb4-4fbf-9aa4-c1c816dfebf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:52.147462 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4c9de-8fb4-4fbf-9aa4-c1c816dfebf9 admin admin] PATH_INFO: `/v3/projects/cf064c0854864e6ea1e96ae7dccdbfbc` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:52.159293 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4c9de-8fb4-4fbf-9aa4-c1c816dfebf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf064c0854864e6ea1e96ae7dccdbfbc)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:52.161252 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b4c9de-8fb4-4fbf-9aa4-c1c816dfebf9 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:52.280026 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23ae69-ee6a-41dd-97f7-21a635c23c76 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:52.287482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3bb3e-fd36-4eca-9a04-c36688626aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:52.289820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3bb3e-fd36-4eca-9a04-c36688626aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:52.291679 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3bb3e-fd36-4eca-9a04-c36688626aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:52.292504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3bb3e-fd36-4eca-9a04-c36688626aee admin admin] PATH_INFO: `/v3/users/f4ef3cbbbe624be3a8b294732b0bf90a` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:52.294665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3bb3e-fd36-4eca-9a04-c36688626aee admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4ef3cbbbe624be3a8b294732b0bf90a)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:52.298474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3bb3e-fd36-4eca-9a04-c36688626aee admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:52.337240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-21b4c9de-8fb4-4fbf-9aa4-c1c816dfebf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf064c0854864e6ea1e96ae7dccdbfbc', 'request_id': 'req-21b4c9de-8fb4-4fbf-9aa4-c1c816dfebf9'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:52.362443 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-21b4c9de-8fb4-4fbf-9aa4-c1c816dfebf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf064c0854864e6ea1e96ae7dccdbfbc', 'request_id': 'req-21b4c9de-8fb4-4fbf-9aa4-c1c816dfebf9'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:52.400134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-21b4c9de-8fb4-4fbf-9aa4-c1c816dfebf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf064c0854864e6ea1e96ae7dccdbfbc', 'request_id': 'req-21b4c9de-8fb4-4fbf-9aa4-c1c816dfebf9'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:52.417613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-20a3bb3e-fd36-4eca-9a04-c36688626aee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4ef3cbbbe624be3a8b294732b0bf90a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:52.423134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-21b4c9de-8fb4-4fbf-9aa4-c1c816dfebf9 admin admin] The token cache is being invalidate because project cf064c0854864e6ea1e96ae7dccdbfbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:12:52.423650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-21b4c9de-8fb4-4fbf-9aa4-c1c816dfebf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:52.425236 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-21b4c9de-8fb4-4fbf-9aa4-c1c816dfebf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:52.428924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 859/1714] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:12:51 2022] DELETE /identity/v3/projects/cf064c0854864e6ea1e96ae7dccdbfbc => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:52.434999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7de2f-23a5-4419-b5ff-3e04ec266490 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:52.435440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7de2f-23a5-4419-b5ff-3e04ec266490 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:52.435888 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7de2f-23a5-4419-b5ff-3e04ec266490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:52.457333 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-20a3bb3e-fd36-4eca-9a04-c36688626aee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4ef3cbbbe624be3a8b294732b0bf90a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:52.474697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-20a3bb3e-fd36-4eca-9a04-c36688626aee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4ef3cbbbe624be3a8b294732b0bf90a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:52.478108 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-20a3bb3e-fd36-4eca-9a04-c36688626aee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4ef3cbbbe624be3a8b294732b0bf90a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:52.491425 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-92f7de2f-23a5-4419-b5ff-3e04ec266490 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:52.493990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-92f7de2f-23a5-4419-b5ff-3e04ec266490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:52.498449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-20a3bb3e-fd36-4eca-9a04-c36688626aee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4ef3cbbbe624be3a8b294732b0bf90a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:52.513678 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 856/1715] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:12:52 2022] DELETE /identity/v3/users/f4ef3cbbbe624be3a8b294732b0bf90a => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:52.586887 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:52.662400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 860/1716] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:52.673137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a981a84-376d-4e89-9517-853da88e50f8 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:52.673430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a981a84-376d-4e89-9517-853da88e50f8 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:52.673718 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a981a84-376d-4e89-9517-853da88e50f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:52.704922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d561b-d60a-4fa7-8640-267fc44d1ac5 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:52.705152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d561b-d60a-4fa7-8640-267fc44d1ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:52.705377 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d561b-d60a-4fa7-8640-267fc44d1ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:52.731333 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-9a981a84-376d-4e89-9517-853da88e50f8 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:52.732890 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9a981a84-376d-4e89-9517-853da88e50f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:52.759083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-986d561b-d60a-4fa7-8640-267fc44d1ac5 None None] MFA Rules not processed for user `9a020687e95a40f88d521757d8b25619`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:52.761831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-986d561b-d60a-4fa7-8640-267fc44d1ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:52.921934 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 861/1717] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:52.938326 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d561b-d60a-4fa7-8640-267fc44d1ac5 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:52.940870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-986d561b-d60a-4fa7-8640-267fc44d1ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:52.960935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 857/1718] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:52.963112 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a981a84-376d-4e89-9517-853da88e50f8 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:52.990759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a981a84-376d-4e89-9517-853da88e50f8 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:53.004260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364fbde-cb42-4652-856f-43fd8d7b14d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:53.004868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364fbde-cb42-4652-856f-43fd8d7b14d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:53.005162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364fbde-cb42-4652-856f-43fd8d7b14d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:53.005844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364fbde-cb42-4652-856f-43fd8d7b14d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:53.006604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364fbde-cb42-4652-856f-43fd8d7b14d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:12:53.011759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364fbde-cb42-4652-856f-43fd8d7b14d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:53.014409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364fbde-cb42-4652-856f-43fd8d7b14d5 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:53.046423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 858/1719] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:12:53.053901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364fbde-cb42-4652-856f-43fd8d7b14d5 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:53.070165 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:53.085007 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364fbde-cb42-4652-856f-43fd8d7b14d5 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:53.092931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77721228-c868-40e3-aaa8-572c438a3069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:53.094031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77721228-c868-40e3-aaa8-572c438a3069 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:53.094031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77721228-c868-40e3-aaa8-572c438a3069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:53.094253 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77721228-c868-40e3-aaa8-572c438a3069 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:53.095052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77721228-c868-40e3-aaa8-572c438a3069 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:53.097791 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77721228-c868-40e3-aaa8-572c438a3069 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:53.154513 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 859/1720] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:53 2022] POST /identity/v3/projects => generated 405 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:53.155529 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77721228-c868-40e3-aaa8-572c438a3069 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:53.157325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d561b-d60a-4fa7-8640-267fc44d1ac5 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:53.269773 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:53.279667 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77721228-c868-40e3-aaa8-572c438a3069 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:53.285341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d0dc8-27d8-4f62-9751-125e7503ab28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:53.286043 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d0dc8-27d8-4f62-9751-125e7503ab28 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:53.286331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d0dc8-27d8-4f62-9751-125e7503ab28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:53.286616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d0dc8-27d8-4f62-9751-125e7503ab28 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:53.287348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d0dc8-27d8-4f62-9751-125e7503ab28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:53.288657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57006ac-4162-4252-9f80-4f4e7fe16eef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:53.289964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d0dc8-27d8-4f62-9751-125e7503ab28 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:53.291542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57006ac-4162-4252-9f80-4f4e7fe16eef service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:53.292668 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57006ac-4162-4252-9f80-4f4e7fe16eef service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:53.296065 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57006ac-4162-4252-9f80-4f4e7fe16eef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:53.330721 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 860/1721] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:53 2022] POST /identity/v3/users => generated 303 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:53.332261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e1a46-8544-4511-b7be-fba5b2ea29dc None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:53.332549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e1a46-8544-4511-b7be-fba5b2ea29dc None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:53.332831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e1a46-8544-4511-b7be-fba5b2ea29dc None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:53.335719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 861/1722] 10.222.0.44 () {54 vars in 912 bytes} [Wed Jul 20 09:12:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:12:53.339212 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e1a46-8544-4511-b7be-fba5b2ea29dc None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:53.372707 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e1a46-8544-4511-b7be-fba5b2ea29dc None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:53.379644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c743dbb-15a3-4c23-abf8-aee86e48ecec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:53.381002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c743dbb-15a3-4c23-abf8-aee86e48ecec admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:53.386356 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c743dbb-15a3-4c23-abf8-aee86e48ecec admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:53.387060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c743dbb-15a3-4c23-abf8-aee86e48ecec admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:53.411824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c743dbb-15a3-4c23-abf8-aee86e48ecec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:53.413512 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c743dbb-15a3-4c23-abf8-aee86e48ecec admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:53.434724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57006ac-4162-4252-9f80-4f4e7fe16eef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:53.436348 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57006ac-4162-4252-9f80-4f4e7fe16eef service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:53.442575 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 862/1723] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:53.445176 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946547c-fc0a-421c-b84f-a7b384e9c58b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:53.445471 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946547c-fc0a-421c-b84f-a7b384e9c58b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:53.445946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946547c-fc0a-421c-b84f-a7b384e9c58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:53.461745 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 862/1724] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:53.469855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b7c9f-2583-474e-9375-32eb26edf4d0 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:53.470298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b7c9f-2583-474e-9375-32eb26edf4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:53.470740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b7c9f-2583-474e-9375-32eb26edf4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:53.519923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-fb9b7c9f-2583-474e-9375-32eb26edf4d0 None None] MFA Rules not processed for user `f0547075e10c498b8d13e1859197cea0`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:53.521521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-fb9b7c9f-2583-474e-9375-32eb26edf4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:53.523682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-8946547c-fc0a-421c-b84f-a7b384e9c58b None None] MFA Rules not processed for user `5e5ad4e122ae471cad4cf1b0e3111dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:53.525802 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8946547c-fc0a-421c-b84f-a7b384e9c58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:53.687524 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 863/1725] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jul 20 09:12:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:53.689106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946547c-fc0a-421c-b84f-a7b384e9c58b None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:53.718806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946547c-fc0a-421c-b84f-a7b384e9c58b None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:53.719806 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 863/1726] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:53.739055 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b4ec7-4815-4b5e-8739-0bafa2a35024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:53.739749 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b4ec7-4815-4b5e-8739-0bafa2a35024 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:53.740033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b4ec7-4815-4b5e-8739-0bafa2a35024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:53.740311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b4ec7-4815-4b5e-8739-0bafa2a35024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:53.740900 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b7c9f-2583-474e-9375-32eb26edf4d0 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:53.741050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b4ec7-4815-4b5e-8739-0bafa2a35024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:53.743046 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b4ec7-4815-4b5e-8739-0bafa2a35024 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:53.773599 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 864/1727] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:53.777820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b4ec7-4815-4b5e-8739-0bafa2a35024 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:53.786440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b7c9f-2583-474e-9375-32eb26edf4d0 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:53.802347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8de4a8-9efa-4f14-b3f6-1fa079647423 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:53.806593 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8de4a8-9efa-4f14-b3f6-1fa079647423 service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:53.807122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8de4a8-9efa-4f14-b3f6-1fa079647423 service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:53.807438 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8de4a8-9efa-4f14-b3f6-1fa079647423 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:53.807736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b4ec7-4815-4b5e-8739-0bafa2a35024 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:53.828662 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5bef6b-0fcb-46cd-bf05-13b4c0e0a025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:53.832357 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bef6b-0fcb-46cd-bf05-13b4c0e0a025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:53.832648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bef6b-0fcb-46cd-bf05-13b4c0e0a025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:53.832936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bef6b-0fcb-46cd-bf05-13b4c0e0a025 admin admin] PATH_INFO: `/v3/projects/90c3385d787e48f1b24a9413a458ff57/users/7b698809b84e4b2082b9d4f5a03ef500/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:53.846870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8de4a8-9efa-4f14-b3f6-1fa079647423 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:53.848510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8de4a8-9efa-4f14-b3f6-1fa079647423 service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:53.881700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 864/1728] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:53.890475 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5bef6b-0fcb-46cd-bf05-13b4c0e0a025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c3385d787e48f1b24a9413a458ff57, user_id=7b698809b84e4b2082b9d4f5a03ef500, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:53.899964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5bef6b-0fcb-46cd-bf05-13b4c0e0a025 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:53.943423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 865/1729] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:53 2022] PUT /identity/v3/projects/90c3385d787e48f1b24a9413a458ff57/users/7b698809b84e4b2082b9d4f5a03ef500/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:53.949337 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c14ede-5ac6-4d62-a507-2700d3e3dba3 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:53.949620 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c14ede-5ac6-4d62-a507-2700d3e3dba3 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:53.949895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c14ede-5ac6-4d62-a507-2700d3e3dba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:53.984555 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-60c14ede-5ac6-4d62-a507-2700d3e3dba3 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:53.986215 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-60c14ede-5ac6-4d62-a507-2700d3e3dba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:54.141180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 865/1730] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:54.188398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5bef6b-0fcb-46cd-bf05-13b4c0e0a025 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:54.191042 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-fd5bef6b-0fcb-46cd-bf05-13b4c0e0a025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:54.243700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:54.369307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5bef6b-0fcb-46cd-bf05-13b4c0e0a025 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:54.377149 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123c5f0-d694-487c-b473-2d917043aa20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:54.380079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123c5f0-d694-487c-b473-2d917043aa20 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:54.386969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123c5f0-d694-487c-b473-2d917043aa20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:54.387595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123c5f0-d694-487c-b473-2d917043aa20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:54.397164 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c14ede-5ac6-4d62-a507-2700d3e3dba3 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:54.398788 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-60c14ede-5ac6-4d62-a507-2700d3e3dba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:54.427079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123c5f0-d694-487c-b473-2d917043aa20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:54.430772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123c5f0-d694-487c-b473-2d917043aa20 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:54.431079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c14ede-5ac6-4d62-a507-2700d3e3dba3 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:54.442766 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de0702-5fc0-4f9b-a9e9-aab65b592026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:54.443451 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de0702-5fc0-4f9b-a9e9-aab65b592026 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:54.443737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de0702-5fc0-4f9b-a9e9-aab65b592026 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:54.444020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de0702-5fc0-4f9b-a9e9-aab65b592026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:54.481141 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de0702-5fc0-4f9b-a9e9-aab65b592026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:54.485272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de0702-5fc0-4f9b-a9e9-aab65b592026 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:54.507093 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 866/1731] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:12:54.508064 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de0702-5fc0-4f9b-a9e9-aab65b592026 service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:54.508653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 866/1732] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:54.509295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-16de0702-5fc0-4f9b-a9e9-aab65b592026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:54.559609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:54.578394 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de0702-5fc0-4f9b-a9e9-aab65b592026 service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:54.598732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1244f-5154-493f-ab9e-73b760e075b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:54.602145 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1244f-5154-493f-ab9e-73b760e075b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:54.602440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1244f-5154-493f-ab9e-73b760e075b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:54.604473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1244f-5154-493f-ab9e-73b760e075b1 admin admin] PATH_INFO: `/v3/projects/9bcbe192603343a89489976323eb5205` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:54.606354 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1244f-5154-493f-ab9e-73b760e075b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bcbe192603343a89489976323eb5205)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:54.625501 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1244f-5154-493f-ab9e-73b760e075b1 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:54.748305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-c2b1244f-5154-493f-ab9e-73b760e075b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bcbe192603343a89489976323eb5205', 'request_id': 'req-c2b1244f-5154-493f-ab9e-73b760e075b1'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:54.763607 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-c2b1244f-5154-493f-ab9e-73b760e075b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bcbe192603343a89489976323eb5205', 'request_id': 'req-c2b1244f-5154-493f-ab9e-73b760e075b1'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:54.778620 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-c2b1244f-5154-493f-ab9e-73b760e075b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bcbe192603343a89489976323eb5205', 'request_id': 'req-c2b1244f-5154-493f-ab9e-73b760e075b1'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:54.798812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-c2b1244f-5154-493f-ab9e-73b760e075b1 admin admin] The token cache is being invalidate because project 9bcbe192603343a89489976323eb5205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:12:54.798812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-c2b1244f-5154-493f-ab9e-73b760e075b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:54.799628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-c2b1244f-5154-493f-ab9e-73b760e075b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:12:54.808200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 867/1733] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:12:54 2022] DELETE /identity/v3/projects/9bcbe192603343a89489976323eb5205 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:54.818798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73c925-34a3-42f9-ad77-bc59c70bab83 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:54.820146 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73c925-34a3-42f9-ad77-bc59c70bab83 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:54.822534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73c925-34a3-42f9-ad77-bc59c70bab83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:54.869603 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-cb73c925-34a3-42f9-ad77-bc59c70bab83 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:54.873109 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-cb73c925-34a3-42f9-ad77-bc59c70bab83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:55.096671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 867/1734] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:55.106968 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b7dd2-bd1e-49fd-9bd1-44fddfa8c112 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:55.107260 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b7dd2-bd1e-49fd-9bd1-44fddfa8c112 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:55.107533 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b7dd2-bd1e-49fd-9bd1-44fddfa8c112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:55.141965 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-034b7dd2-bd1e-49fd-9bd1-44fddfa8c112 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:55.143808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-034b7dd2-bd1e-49fd-9bd1-44fddfa8c112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:55.170049 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 868/1735] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:55.171995 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73c925-34a3-42f9-ad77-bc59c70bab83 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:55.190535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73c925-34a3-42f9-ad77-bc59c70bab83 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:55.206985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7988021-07bb-42b0-bdde-03abb67f67aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:55.208094 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7988021-07bb-42b0-bdde-03abb67f67aa admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:55.208497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7988021-07bb-42b0-bdde-03abb67f67aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:55.208890 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7988021-07bb-42b0-bdde-03abb67f67aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:55.209770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7988021-07bb-42b0-bdde-03abb67f67aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:12:55.210193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7988021-07bb-42b0-bdde-03abb67f67aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:55.214376 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7988021-07bb-42b0-bdde-03abb67f67aa admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:55.252839 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 868/1736] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:12:55.254980 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b7dd2-bd1e-49fd-9bd1-44fddfa8c112 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:55.270927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb3bab-fcf3-4d74-bbb3-01e98b6ad2ca None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:55.271366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb3bab-fcf3-4d74-bbb3-01e98b6ad2ca None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:55.271786 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb3bab-fcf3-4d74-bbb3-01e98b6ad2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:55.280913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b7dd2-bd1e-49fd-9bd1-44fddfa8c112 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:55.310144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b30518-91d5-42c9-887d-41e64453c66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:55.310864 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b30518-91d5-42c9-887d-41e64453c66a admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:55.311147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b30518-91d5-42c9-887d-41e64453c66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:55.311421 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b30518-91d5-42c9-887d-41e64453c66a admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:55.312125 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b30518-91d5-42c9-887d-41e64453c66a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:55.329047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b30518-91d5-42c9-887d-41e64453c66a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:55.342578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-2adb3bab-fcf3-4d74-bbb3-01e98b6ad2ca None None] MFA Rules not processed for user `a24a58daa8ed4d77a649a670cc358638`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:55.343868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2adb3bab-fcf3-4d74-bbb3-01e98b6ad2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:55.442486 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 869/1737] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:55 2022] POST /identity/v3/projects => generated 365 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:55.444475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b30518-91d5-42c9-887d-41e64453c66a admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:55.545933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 869/1738] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:12:55.565257 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab22aa-d86a-4ce4-92fe-84b95bd1eb8f None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:55.566174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab22aa-d86a-4ce4-92fe-84b95bd1eb8f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:55.566672 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab22aa-d86a-4ce4-92fe-84b95bd1eb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:55.656361 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-e7ab22aa-d86a-4ce4-92fe-84b95bd1eb8f None None] MFA Rules not processed for user `a24a58daa8ed4d77a649a670cc358638`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:55.657617 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e7ab22aa-d86a-4ce4-92fe-84b95bd1eb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:55.687932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b30518-91d5-42c9-887d-41e64453c66a admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:55.701659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d8669-45b1-4c0c-95ea-afe63132b76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:55.702530 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d8669-45b1-4c0c-95ea-afe63132b76b admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:55.702977 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d8669-45b1-4c0c-95ea-afe63132b76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:55.703369 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d8669-45b1-4c0c-95ea-afe63132b76b admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:55.708626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d8669-45b1-4c0c-95ea-afe63132b76b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:55.721689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d8669-45b1-4c0c-95ea-afe63132b76b admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:55.789594 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 870/1739] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:55 2022] POST /identity/v3/users => generated 283 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:55.793961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:55.794823 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d8669-45b1-4c0c-95ea-afe63132b76b admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:55.828354 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d8669-45b1-4c0c-95ea-afe63132b76b admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:55.845593 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63177d-f2cc-4e6b-90b4-ad9564efe945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:55.846880 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63177d-f2cc-4e6b-90b4-ad9564efe945 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:55.855088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63177d-f2cc-4e6b-90b4-ad9564efe945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:55.855782 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63177d-f2cc-4e6b-90b4-ad9564efe945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:55.856749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63177d-f2cc-4e6b-90b4-ad9564efe945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:55.859661 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63177d-f2cc-4e6b-90b4-ad9564efe945 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:55.889535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 870/1740] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:12:55.910275 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab22aa-d86a-4ce4-92fe-84b95bd1eb8f None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:55.911098 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 871/1741] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:55.912064 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63177d-f2cc-4e6b-90b4-ad9564efe945 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:55.914594 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e7ab22aa-d86a-4ce4-92fe-84b95bd1eb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:55.958431 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63177d-f2cc-4e6b-90b4-ad9564efe945 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:55.976176 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a158f-7079-4999-bb48-ab4a3967b4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:55.977055 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a158f-7079-4999-bb48-ab4a3967b4cf admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:55.977415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a158f-7079-4999-bb48-ab4a3967b4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:55.977801 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a158f-7079-4999-bb48-ab4a3967b4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:55.979403 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a158f-7079-4999-bb48-ab4a3967b4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:55.984404 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a158f-7079-4999-bb48-ab4a3967b4cf admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:56.011642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 872/1742] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:56.016423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a158f-7079-4999-bb48-ab4a3967b4cf admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:56.037328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a158f-7079-4999-bb48-ab4a3967b4cf admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:56.053286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236380bc-cd5d-4e81-8310-a904d3490f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:56.057324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236380bc-cd5d-4e81-8310-a904d3490f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:56.057662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236380bc-cd5d-4e81-8310-a904d3490f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:56.057896 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236380bc-cd5d-4e81-8310-a904d3490f11 admin admin] PATH_INFO: `/v3/projects/38111352942d40be8a50621adc1cff4f/users/8bfbbba3ae0b41aebaf6e44acada71f2/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:56.094045 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236380bc-cd5d-4e81-8310-a904d3490f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38111352942d40be8a50621adc1cff4f, user_id=8bfbbba3ae0b41aebaf6e44acada71f2, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:56.101471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236380bc-cd5d-4e81-8310-a904d3490f11 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:56.117062 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:56.125150 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 873/1743] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:56 2022] PUT /identity/v3/projects/38111352942d40be8a50621adc1cff4f/users/8bfbbba3ae0b41aebaf6e44acada71f2/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:56.126806 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97396d1a-4473-44cc-b3b9-32551d3a6d4e None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:56.127672 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97396d1a-4473-44cc-b3b9-32551d3a6d4e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:56.128133 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97396d1a-4473-44cc-b3b9-32551d3a6d4e None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:56.143324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab22aa-d86a-4ce4-92fe-84b95bd1eb8f None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:56.147295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 874/1744] 10.222.0.44 () {54 vars in 961 bytes} [Wed Jul 20 09:12:56 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:12:56.149284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf121765-c3cd-40ef-bcf3-9628b666eda2 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:56.149808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf121765-c3cd-40ef-bcf3-9628b666eda2 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:56.151028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf121765-c3cd-40ef-bcf3-9628b666eda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:56.189360 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-bf121765-c3cd-40ef-bcf3-9628b666eda2 None None] MFA Rules not processed for user `8bfbbba3ae0b41aebaf6e44acada71f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:56.191373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-bf121765-c3cd-40ef-bcf3-9628b666eda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:56.238468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d2378-b89f-4739-9aa5-e572035a6d2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:56.239457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d2378-b89f-4739-9aa5-e572035a6d2f service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:56.240094 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d2378-b89f-4739-9aa5-e572035a6d2f service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:56.240479 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d2378-b89f-4739-9aa5-e572035a6d2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:56.342543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 875/1745] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:56.345153 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2e77b-8c20-4c74-83fa-53fd895dc309 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:56.345936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d2378-b89f-4739-9aa5-e572035a6d2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:56.347057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2e77b-8c20-4c74-83fa-53fd895dc309 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:56.347848 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d2378-b89f-4739-9aa5-e572035a6d2f service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:56.348072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2e77b-8c20-4c74-83fa-53fd895dc309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:56.363721 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 871/1746] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:56.365500 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff3938-6071-42e7-85e5-d1eb0ff4f673 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:56.366177 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff3938-6071-42e7-85e5-d1eb0ff4f673 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:56.366569 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff3938-6071-42e7-85e5-d1eb0ff4f673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:56.394138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-51e2e77b-8c20-4c74-83fa-53fd895dc309 None None] MFA Rules not processed for user `abdb339c5f5740d6aff4dfd1066f9384`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:56.396686 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-deff3938-6071-42e7-85e5-d1eb0ff4f673 None None] MFA Rules not processed for user `066f69a87f5f48c7b20c1e0fcf5c1107`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:56.397555 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-51e2e77b-8c20-4c74-83fa-53fd895dc309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:56.399715 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-deff3938-6071-42e7-85e5-d1eb0ff4f673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:56.420903 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:56.503954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 872/1747] 10.222.0.44 () {58 vars in 1058 bytes} [Wed Jul 20 09:12:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:56.505599 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff3938-6071-42e7-85e5-d1eb0ff4f673 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:56.594984 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 876/1748] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:56.600050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2e77b-8c20-4c74-83fa-53fd895dc309 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:56.624859 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff3938-6071-42e7-85e5-d1eb0ff4f673 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:56.627863 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2e77b-8c20-4c74-83fa-53fd895dc309 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:56.631090 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d24c2-8c8c-42b5-a69a-f4bf0c2a118a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:56.631771 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d24c2-8c8c-42b5-a69a-f4bf0c2a118a admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:56.632055 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d24c2-8c8c-42b5-a69a-f4bf0c2a118a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:56.632330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d24c2-8c8c-42b5-a69a-f4bf0c2a118a admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:56.633029 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d24c2-8c8c-42b5-a69a-f4bf0c2a118a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:56.636415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d24c2-8c8c-42b5-a69a-f4bf0c2a118a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:56.637315 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341fa99-6ec8-47b7-adc3-fd7de2ee90d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '066f69a87f5f48c7b20c1e0fcf5c1107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:56.638407 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341fa99-6ec8-47b7-adc3-fd7de2ee90d4 service placement] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:56.638904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341fa99-6ec8-47b7-adc3-fd7de2ee90d4 service placement] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:56.639292 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341fa99-6ec8-47b7-adc3-fd7de2ee90d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:56.641630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a341fa99-6ec8-47b7-adc3-fd7de2ee90d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:56.663639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341fa99-6ec8-47b7-adc3-fd7de2ee90d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:56.666295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341fa99-6ec8-47b7-adc3-fd7de2ee90d4 service placement] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:56.685138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 873/1749] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:56 2022] POST /identity/v3/projects => generated 363 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:56.687774 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 877/1750] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:56.689177 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d24c2-8c8c-42b5-a69a-f4bf0c2a118a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:56.854335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d24c2-8c8c-42b5-a69a-f4bf0c2a118a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:56.867449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5975e-825a-4004-9221-2045d6ddff50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:56.868158 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5975e-825a-4004-9221-2045d6ddff50 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:56.868448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5975e-825a-4004-9221-2045d6ddff50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:56.869970 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5975e-825a-4004-9221-2045d6ddff50 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:56.871337 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5975e-825a-4004-9221-2045d6ddff50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:56.874126 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5975e-825a-4004-9221-2045d6ddff50 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:56.932127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 874/1751] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:56 2022] POST /identity/v3/users => generated 288 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:56.937108 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341fa99-6ec8-47b7-adc3-fd7de2ee90d4 service placement] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:56.956364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341fa99-6ec8-47b7-adc3-fd7de2ee90d4 service placement] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:56.971957 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30ec87-f68d-49b3-b130-a6c74cf114a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:56.974003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30ec87-f68d-49b3-b130-a6c74cf114a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:56.974821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30ec87-f68d-49b3-b130-a6c74cf114a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:56.975322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30ec87-f68d-49b3-b130-a6c74cf114a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:56.976262 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30ec87-f68d-49b3-b130-a6c74cf114a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:56.978617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30ec87-f68d-49b3-b130-a6c74cf114a9 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:57.012815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 878/1752] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:57.017216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5975e-825a-4004-9221-2045d6ddff50 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:57.037555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5975e-825a-4004-9221-2045d6ddff50 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:57.061454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419f3d5-f6bc-4f62-b7e8-ea5203380702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:57.062631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419f3d5-f6bc-4f62-b7e8-ea5203380702 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:57.063004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419f3d5-f6bc-4f62-b7e8-ea5203380702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:57.063295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419f3d5-f6bc-4f62-b7e8-ea5203380702 admin admin] PATH_INFO: `/v3/projects/c8fa54f78c5341e69c04f3f61cdb5eba/users/c6f5518ec7314a5eabcb8655993a42ef/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:57.098517 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419f3d5-f6bc-4f62-b7e8-ea5203380702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8fa54f78c5341e69c04f3f61cdb5eba, user_id=c6f5518ec7314a5eabcb8655993a42ef, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:57.101379 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419f3d5-f6bc-4f62-b7e8-ea5203380702 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:57.144115 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 875/1753] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:57 2022] PUT /identity/v3/projects/c8fa54f78c5341e69c04f3f61cdb5eba/users/c6f5518ec7314a5eabcb8655993a42ef/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:57.146147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30ec87-f68d-49b3-b130-a6c74cf114a9 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:57.162044 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30ec87-f68d-49b3-b130-a6c74cf114a9 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:57.283520 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c0dd0-8795-420e-9ab1-4d0fda2de047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:57.284293 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c0dd0-8795-420e-9ab1-4d0fda2de047 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:57.284729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c0dd0-8795-420e-9ab1-4d0fda2de047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:57.285018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c0dd0-8795-420e-9ab1-4d0fda2de047 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:57.285820 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c0dd0-8795-420e-9ab1-4d0fda2de047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:57.290264 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c0dd0-8795-420e-9ab1-4d0fda2de047 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:57.311684 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 879/1754] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:57.311874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419f3d5-f6bc-4f62-b7e8-ea5203380702 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:57.327166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419f3d5-f6bc-4f62-b7e8-ea5203380702 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:57.334811 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac4e16-59b4-44e0-84a5-ecb97a190dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:57.335902 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac4e16-59b4-44e0-84a5-ecb97a190dff admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:57.336225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac4e16-59b4-44e0-84a5-ecb97a190dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:57.336568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac4e16-59b4-44e0-84a5-ecb97a190dff admin admin] PATH_INFO: `/v3/projects/c8fa54f78c5341e69c04f3f61cdb5eba/users/c6f5518ec7314a5eabcb8655993a42ef/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:57.342862 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac4e16-59b4-44e0-84a5-ecb97a190dff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8fa54f78c5341e69c04f3f61cdb5eba, user_id=c6f5518ec7314a5eabcb8655993a42ef, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:57.347520 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac4e16-59b4-44e0-84a5-ecb97a190dff admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:57.350530 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:57.409048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4122320-143e-4efe-8306-b64512cb94b6 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:57.409370 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4122320-143e-4efe-8306-b64512cb94b6 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:57.409665 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4122320-143e-4efe-8306-b64512cb94b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:57.418962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 876/1755] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:57 2022] PUT /identity/v3/projects/c8fa54f78c5341e69c04f3f61cdb5eba/users/c6f5518ec7314a5eabcb8655993a42ef/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:12:57.431433 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:57.463336 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-f4122320-143e-4efe-8306-b64512cb94b6 None None] MFA Rules not processed for user `c6f5518ec7314a5eabcb8655993a42ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:57.464738 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f4122320-143e-4efe-8306-b64512cb94b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:57.516850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e1662-4ea0-4f63-bf01-27d6333777c0 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:57.517146 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e1662-4ea0-4f63-bf01-27d6333777c0 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:57.517431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e1662-4ea0-4f63-bf01-27d6333777c0 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:57.518934 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 877/1756] 10.222.0.44 () {54 vars in 912 bytes} [Wed Jul 20 09:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 09:12:57.528443 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854e22f-f8da-48aa-971d-d606e5df160a None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:57.528737 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854e22f-f8da-48aa-971d-d606e5df160a None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:57.529021 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854e22f-f8da-48aa-971d-d606e5df160a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:57.602411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-4854e22f-f8da-48aa-971d-d606e5df160a None None] MFA Rules not processed for user `5e5ad4e122ae471cad4cf1b0e3111dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:57.606964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-4854e22f-f8da-48aa-971d-d606e5df160a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:57.696515 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 880/1757] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:57 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:57.719209 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4122320-143e-4efe-8306-b64512cb94b6 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:57.721356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f4122320-143e-4efe-8306-b64512cb94b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:57.773176 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 878/1758] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jul 20 09:12:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:57.871705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4122320-143e-4efe-8306-b64512cb94b6 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:57.874394 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854e22f-f8da-48aa-971d-d606e5df160a None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:57.876153 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-4854e22f-f8da-48aa-971d-d606e5df160a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:57.883490 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6195d-b38b-4f6f-bca1-ef954126bcc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:57.887485 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6195d-b38b-4f6f-bca1-ef954126bcc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:57.887823 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6195d-b38b-4f6f-bca1-ef954126bcc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:57.888169 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6195d-b38b-4f6f-bca1-ef954126bcc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:57.893235 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854e22f-f8da-48aa-971d-d606e5df160a None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:57.902371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96819a2-10e7-4639-917a-28f2aa9b9cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:57.904261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96819a2-10e7-4639-917a-28f2aa9b9cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:57.906327 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96819a2-10e7-4639-917a-28f2aa9b9cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:57.906618 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96819a2-10e7-4639-917a-28f2aa9b9cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:57.934349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6195d-b38b-4f6f-bca1-ef954126bcc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:57.935267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96819a2-10e7-4639-917a-28f2aa9b9cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:57.937947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96819a2-10e7-4639-917a-28f2aa9b9cd1 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:57.938103 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6195d-b38b-4f6f-bca1-ef954126bcc3 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:57.967465 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 879/1759] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:12:57.976933 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 881/1760] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:57 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:12:58.204667 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60e7bf-6246-4eb5-b219-25ef30af45ea None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:58.205163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60e7bf-6246-4eb5-b219-25ef30af45ea None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:58.206203 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60e7bf-6246-4eb5-b219-25ef30af45ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:58.253325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6195d-b38b-4f6f-bca1-ef954126bcc3 service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:58.254655 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-5d60e7bf-6246-4eb5-b219-25ef30af45ea None None] MFA Rules not processed for user `7b698809b84e4b2082b9d4f5a03ef500`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:12:58.255187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-cea6195d-b38b-4f6f-bca1-ef954126bcc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:58.268120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5d60e7bf-6246-4eb5-b219-25ef30af45ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:58.300094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6195d-b38b-4f6f-bca1-ef954126bcc3 service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:58.325422 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be3cd5-9ad4-4d85-acf7-8df30a8307f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:58.327407 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be3cd5-9ad4-4d85-acf7-8df30a8307f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:58.337382 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be3cd5-9ad4-4d85-acf7-8df30a8307f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:58.337783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be3cd5-9ad4-4d85-acf7-8df30a8307f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:58.506598 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 880/1761] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:58 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:12:58.512909 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60e7bf-6246-4eb5-b219-25ef30af45ea None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:58.518169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5d60e7bf-6246-4eb5-b219-25ef30af45ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:12:58.598011 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be3cd5-9ad4-4d85-acf7-8df30a8307f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:58.608037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be3cd5-9ad4-4d85-acf7-8df30a8307f2 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:58.668269 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 882/1762] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:12:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:12:58.686032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60e7bf-6246-4eb5-b219-25ef30af45ea None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:58.702891 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afbfc07-f509-4050-a981-9ed2042f5d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:58.703619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbfc07-f509-4050-a981-9ed2042f5d58 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:58.704004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbfc07-f509-4050-a981-9ed2042f5d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:58.704167 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbfc07-f509-4050-a981-9ed2042f5d58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:58.704871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afbfc07-f509-4050-a981-9ed2042f5d58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:58.709055 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afbfc07-f509-4050-a981-9ed2042f5d58 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:58.830948 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 881/1763] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:12:58 2022] POST /identity/v3/projects => generated 405 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:58.832288 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be3cd5-9ad4-4d85-acf7-8df30a8307f2 service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:58.886046 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be3cd5-9ad4-4d85-acf7-8df30a8307f2 service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:59.052733 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:59.182631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d565d-c4dc-4112-a9e7-14c179a6ffeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:59.200028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d565d-c4dc-4112-a9e7-14c179a6ffeb admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:59.201756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d565d-c4dc-4112-a9e7-14c179a6ffeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:59.204512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d565d-c4dc-4112-a9e7-14c179a6ffeb admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:59.205545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d565d-c4dc-4112-a9e7-14c179a6ffeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:59.214184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d565d-c4dc-4112-a9e7-14c179a6ffeb admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:59.310834 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 883/1764] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:12:58 2022] POST /identity/v3/users => generated 309 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:12:59.312815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afbfc07-f509-4050-a981-9ed2042f5d58 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:59.327321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:12:59.348496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afbfc07-f509-4050-a981-9ed2042f5d58 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:59.371249 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5a4a0-a7e4-49dd-afaa-a0e32c25d19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:59.371994 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5a4a0-a7e4-49dd-afaa-a0e32c25d19a admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:59.372311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5a4a0-a7e4-49dd-afaa-a0e32c25d19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:59.372568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae5a4a0-a7e4-49dd-afaa-a0e32c25d19a admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:59.373321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5a4a0-a7e4-49dd-afaa-a0e32c25d19a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:59.374987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae5a4a0-a7e4-49dd-afaa-a0e32c25d19a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:59.412490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 882/1765] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:59.417615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d565d-c4dc-4112-a9e7-14c179a6ffeb admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:59.430823 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d565d-c4dc-4112-a9e7-14c179a6ffeb admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:59.461035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4c4de-0b5f-4feb-aed9-116358bc8955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:59.483491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4c4de-0b5f-4feb-aed9-116358bc8955 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:59.484198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4c4de-0b5f-4feb-aed9-116358bc8955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:59.485738 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4c4de-0b5f-4feb-aed9-116358bc8955 admin admin] PATH_INFO: `/v3/projects/4779de95d327453c83e06ab6345411de/users/e2760b33d89f48a996f3f93cfc9acf91/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:59.538260 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4c4de-0b5f-4feb-aed9-116358bc8955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4779de95d327453c83e06ab6345411de, user_id=e2760b33d89f48a996f3f93cfc9acf91, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:59.548100 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b4c4de-0b5f-4feb-aed9-116358bc8955 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:59.610024 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 884/1766] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:59 2022] PUT /identity/v3/projects/4779de95d327453c83e06ab6345411de/users/e2760b33d89f48a996f3f93cfc9acf91/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:12:59.613979 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5a4a0-a7e4-49dd-afaa-a0e32c25d19a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:59.638130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae5a4a0-a7e4-49dd-afaa-a0e32c25d19a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:12:59.876022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347f4ca-9995-4c2f-a62e-56d858aa0e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:12:59.883441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347f4ca-9995-4c2f-a62e-56d858aa0e0b admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:59.883842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347f4ca-9995-4c2f-a62e-56d858aa0e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:59.887178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347f4ca-9995-4c2f-a62e-56d858aa0e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:59.888607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347f4ca-9995-4c2f-a62e-56d858aa0e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:12:59.893709 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347f4ca-9995-4c2f-a62e-56d858aa0e0b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:12:59.938962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 883/1767] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:12:59.943729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4c4de-0b5f-4feb-aed9-116358bc8955 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:12:59.990697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381ecca-5bfe-49a5-9d50-1ff716ae37e5 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:12:59.991022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381ecca-5bfe-49a5-9d50-1ff716ae37e5 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:12:59.991310 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381ecca-5bfe-49a5-9d50-1ff716ae37e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:12:59.999067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b4c4de-0b5f-4feb-aed9-116358bc8955 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:00.015800 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd8ebc-5228-4498-8bdc-fe3d0c9e35b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:00.016733 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd8ebc-5228-4498-8bdc-fe3d0c9e35b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:00.017024 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd8ebc-5228-4498-8bdc-fe3d0c9e35b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:00.017308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd8ebc-5228-4498-8bdc-fe3d0c9e35b7 admin admin] PATH_INFO: `/v3/projects/4779de95d327453c83e06ab6345411de/users/e2760b33d89f48a996f3f93cfc9acf91/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:00.029271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd8ebc-5228-4498-8bdc-fe3d0c9e35b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4779de95d327453c83e06ab6345411de, user_id=e2760b33d89f48a996f3f93cfc9acf91, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:00.032592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd8ebc-5228-4498-8bdc-fe3d0c9e35b7 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:00.074028 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-d381ecca-5bfe-49a5-9d50-1ff716ae37e5 None None] MFA Rules not processed for user `9a020687e95a40f88d521757d8b25619`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:00.078885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d381ecca-5bfe-49a5-9d50-1ff716ae37e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:00.084872 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 885/1768] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:12:59 2022] PUT /identity/v3/projects/4779de95d327453c83e06ab6345411de/users/e2760b33d89f48a996f3f93cfc9acf91/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:00.350938 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 884/1769] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:12:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:00.351857 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d640e7-f99c-47ad-b9e6-fbdde16a895b None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:00.352587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d640e7-f99c-47ad-b9e6-fbdde16a895b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:00.353070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d640e7-f99c-47ad-b9e6-fbdde16a895b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:00.420104 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-e8d640e7-f99c-47ad-b9e6-fbdde16a895b None None] MFA Rules not processed for user `f0547075e10c498b8d13e1859197cea0`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:00.423974 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e8d640e7-f99c-47ad-b9e6-fbdde16a895b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:00.508442 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 886/1770] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:01.599179 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041d671-7aa0-46ef-8640-41d9f3015f58 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:01.599758 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041d671-7aa0-46ef-8640-41d9f3015f58 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:01.599758 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041d671-7aa0-46ef-8640-41d9f3015f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:01.720351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-8041d671-7aa0-46ef-8640-41d9f3015f58 None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:01.721585 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8041d671-7aa0-46ef-8640-41d9f3015f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:01.889454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 885/1771] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jul 20 09:13:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:02.161693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d640e7-f99c-47ad-b9e6-fbdde16a895b None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:02.163239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e8d640e7-f99c-47ad-b9e6-fbdde16a895b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:02.479058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d640e7-f99c-47ad-b9e6-fbdde16a895b None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:02.499133 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f7b96-936a-401f-a36e-d9ee14eadde1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:02.500327 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f7b96-936a-401f-a36e-d9ee14eadde1 service swift] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:02.500800 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f7b96-936a-401f-a36e-d9ee14eadde1 service swift] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:02.502165 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f7b96-936a-401f-a36e-d9ee14eadde1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:02.528349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294f7b96-936a-401f-a36e-d9ee14eadde1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:02.536822 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294f7b96-936a-401f-a36e-d9ee14eadde1 service swift] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:02.552205 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:02.567821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 887/1772] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:13:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:13:03.702020 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041d671-7aa0-46ef-8640-41d9f3015f58 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:03.704515 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8041d671-7aa0-46ef-8640-41d9f3015f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:03.725177 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:03.962644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041d671-7aa0-46ef-8640-41d9f3015f58 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:03.977841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc06ed-399c-4f6a-a764-cd75f431ebc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:03.981294 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc06ed-399c-4f6a-a764-cd75f431ebc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:03.981588 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc06ed-399c-4f6a-a764-cd75f431ebc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:03.982988 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc06ed-399c-4f6a-a764-cd75f431ebc1 admin admin] PATH_INFO: `/v3/users/a23ed2a3521a4a328e46eb914938660e` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:03.987861 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc06ed-399c-4f6a-a764-cd75f431ebc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a23ed2a3521a4a328e46eb914938660e)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:03.990994 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc06ed-399c-4f6a-a764-cd75f431ebc1 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:04.122345 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6bdc06ed-399c-4f6a-a764-cd75f431ebc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a23ed2a3521a4a328e46eb914938660e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:04.186044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6bdc06ed-399c-4f6a-a764-cd75f431ebc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a23ed2a3521a4a328e46eb914938660e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:04.219647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6bdc06ed-399c-4f6a-a764-cd75f431ebc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a23ed2a3521a4a328e46eb914938660e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:04.223354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6bdc06ed-399c-4f6a-a764-cd75f431ebc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a23ed2a3521a4a328e46eb914938660e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:04.239924 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6bdc06ed-399c-4f6a-a764-cd75f431ebc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a23ed2a3521a4a328e46eb914938660e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:04.272837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 886/1773] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:03 2022] DELETE /identity/v3/users/a23ed2a3521a4a328e46eb914938660e => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:04.275591 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab76678-d3ef-4e0a-b07d-1d23c4bbfbc8 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:04.276265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab76678-d3ef-4e0a-b07d-1d23c4bbfbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:04.277671 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab76678-d3ef-4e0a-b07d-1d23c4bbfbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:04.326134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-fab76678-d3ef-4e0a-b07d-1d23c4bbfbc8 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:04.332199 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-fab76678-d3ef-4e0a-b07d-1d23c4bbfbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:04.537712 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 888/1774] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:05.005542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc06ed-399c-4f6a-a764-cd75f431ebc1 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:05.007702 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6bdc06ed-399c-4f6a-a764-cd75f431ebc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:05.288517 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc06ed-399c-4f6a-a764-cd75f431ebc1 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:05.302110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c64486-cf4f-49d3-9968-5e880198ef8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:05.305642 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c64486-cf4f-49d3-9968-5e880198ef8b service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:05.309914 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c64486-cf4f-49d3-9968-5e880198ef8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:05.311307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c64486-cf4f-49d3-9968-5e880198ef8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:05.354191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c64486-cf4f-49d3-9968-5e880198ef8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:05.367180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c64486-cf4f-49d3-9968-5e880198ef8b service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:05.434532 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 887/1775] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:06.578807 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab76678-d3ef-4e0a-b07d-1d23c4bbfbc8 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:06.581600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-fab76678-d3ef-4e0a-b07d-1d23c4bbfbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:06.605781 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab76678-d3ef-4e0a-b07d-1d23c4bbfbc8 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:06.611778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:06.613124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:06.613634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:06.614127 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33 admin admin] PATH_INFO: `/v3/projects/167b09014aa3444b94694c2ce89eddab` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:06.615631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=167b09014aa3444b94694c2ce89eddab)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:06.617897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:06.621446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de0da8-1a7e-4238-8a72-8305d57b3d4e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:06.622053 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de0da8-1a7e-4238-8a72-8305d57b3d4e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:06.622053 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de0da8-1a7e-4238-8a72-8305d57b3d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:06.673465 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-83de0da8-1a7e-4238-8a72-8305d57b3d4e None None] MFA Rules not processed for user `e2760b33d89f48a996f3f93cfc9acf91`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:06.675546 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-83de0da8-1a7e-4238-8a72-8305d57b3d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:06.689443 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '167b09014aa3444b94694c2ce89eddab', 'request_id': 'req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:06.697184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '167b09014aa3444b94694c2ce89eddab', 'request_id': 'req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:06.723222 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:06.780336 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '167b09014aa3444b94694c2ce89eddab', 'request_id': 'req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:06.875526 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33 admin admin] The token cache is being invalidate because project 167b09014aa3444b94694c2ce89eddab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:06.879557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:06.882245 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:06.894591 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 889/1776] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:06 2022] DELETE /identity/v3/projects/167b09014aa3444b94694c2ce89eddab => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:06.899170 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:06.903248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:06.980388 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 888/1777] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:06 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:07.116176 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81edd72-cbd3-4668-ae3e-a0112daeabba None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:07.116506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81edd72-cbd3-4668-ae3e-a0112daeabba None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:07.116757 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81edd72-cbd3-4668-ae3e-a0112daeabba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:07.209949 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a2f3a-766b-41e9-ae43-91f3e3580e33 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:07.216482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-b81edd72-cbd3-4668-ae3e-a0112daeabba None None] MFA Rules not processed for user `5e5ad4e122ae471cad4cf1b0e3111dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:07.226472 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b81edd72-cbd3-4668-ae3e-a0112daeabba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:07.256619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458458cf-7d32-4909-ab5a-632ca1eba6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:07.261420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458458cf-7d32-4909-ab5a-632ca1eba6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:07.264673 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458458cf-7d32-4909-ab5a-632ca1eba6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:07.265785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458458cf-7d32-4909-ab5a-632ca1eba6d3 admin admin] PATH_INFO: `/v3/users/6a9cbe19290842169632bda1d2b6a528` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:07.274457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458458cf-7d32-4909-ab5a-632ca1eba6d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a9cbe19290842169632bda1d2b6a528)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:07.282597 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458458cf-7d32-4909-ab5a-632ca1eba6d3 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:07.416162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:07.541947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 889/1778] 10.222.0.74 () {58 vars in 1015 bytes} [Wed Jul 20 09:13:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:07.554345 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-458458cf-7d32-4909-ab5a-632ca1eba6d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a9cbe19290842169632bda1d2b6a528'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:07.566904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81edd72-cbd3-4668-ae3e-a0112daeabba None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:07.569726 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b81edd72-cbd3-4668-ae3e-a0112daeabba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:07.609050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-458458cf-7d32-4909-ab5a-632ca1eba6d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a9cbe19290842169632bda1d2b6a528'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:07.612806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81edd72-cbd3-4668-ae3e-a0112daeabba None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:07.640161 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-458458cf-7d32-4909-ab5a-632ca1eba6d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a9cbe19290842169632bda1d2b6a528'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:07.649150 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-458458cf-7d32-4909-ab5a-632ca1eba6d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a9cbe19290842169632bda1d2b6a528'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:07.658312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2858c51-0868-4670-9987-9b718e5c0707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:07.661684 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2858c51-0868-4670-9987-9b718e5c0707 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:07.662150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2858c51-0868-4670-9987-9b718e5c0707 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:07.662594 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2858c51-0868-4670-9987-9b718e5c0707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:07.682072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-458458cf-7d32-4909-ab5a-632ca1eba6d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a9cbe19290842169632bda1d2b6a528'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:07.699065 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2858c51-0868-4670-9987-9b718e5c0707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:07.702114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2858c51-0868-4670-9987-9b718e5c0707 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:07.711680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 890/1779] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:06 2022] DELETE /identity/v3/users/6a9cbe19290842169632bda1d2b6a528 => generated 0 bytes in 813 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:07.713315 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458458cf-7d32-4909-ab5a-632ca1eba6d3 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:07.723802 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-458458cf-7d32-4909-ab5a-632ca1eba6d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:07.749676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 890/1780] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:07.750553 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2858c51-0868-4670-9987-9b718e5c0707 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:07.755773 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d2858c51-0868-4670-9987-9b718e5c0707 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:07.807314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2858c51-0868-4670-9987-9b718e5c0707 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:07.947028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458458cf-7d32-4909-ab5a-632ca1eba6d3 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:07.956401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f1d3fe-7a99-430e-ac5d-4a26f44f4874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:07.957321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1d3fe-7a99-430e-ac5d-4a26f44f4874 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:07.958187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1d3fe-7a99-430e-ac5d-4a26f44f4874 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:07.958556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1d3fe-7a99-430e-ac5d-4a26f44f4874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:07.962773 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-35f1d3fe-7a99-430e-ac5d-4a26f44f4874 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:08.003383 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f1d3fe-7a99-430e-ac5d-4a26f44f4874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:08.006167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f1d3fe-7a99-430e-ac5d-4a26f44f4874 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:08.024461 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 891/1781] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:08.025860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc9ac9-a766-49a0-9ad1-8aeca7883c2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:08.026565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc9ac9-a766-49a0-9ad1-8aeca7883c2c service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:08.026910 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc9ac9-a766-49a0-9ad1-8aeca7883c2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:08.027828 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc9ac9-a766-49a0-9ad1-8aeca7883c2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:08.047676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc9ac9-a766-49a0-9ad1-8aeca7883c2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:08.049552 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc9ac9-a766-49a0-9ad1-8aeca7883c2c service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:08.072102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 891/1782] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:10.277159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5ae57-78e6-4a98-a87f-c6af42765ba7 None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:10.278057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5ae57-78e6-4a98-a87f-c6af42765ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:10.278504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5ae57-78e6-4a98-a87f-c6af42765ba7 None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:10.280407 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 892/1783] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jul 20 09:13:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:13:10.422186 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f45696-88cb-4ba6-91d8-e195dc854ad0 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:10.427798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f45696-88cb-4ba6-91d8-e195dc854ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:10.427798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f45696-88cb-4ba6-91d8-e195dc854ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:10.458805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-d0f45696-88cb-4ba6-91d8-e195dc854ad0 None None] MFA Rules not processed for user `066f69a87f5f48c7b20c1e0fcf5c1107`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:10.460486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d0f45696-88cb-4ba6-91d8-e195dc854ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:10.499726 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5ae57-78e6-4a98-a87f-c6af42765ba7 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:10.502265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6df5ae57-78e6-4a98-a87f-c6af42765ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:10.731934 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:10.804226 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5ae57-78e6-4a98-a87f-c6af42765ba7 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:10.816308 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 892/1784] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jul 20 09:13:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:10.835800 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acec7f7-8b24-42a0-beeb-bac23aa6660a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:10.838506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acec7f7-8b24-42a0-beeb-bac23aa6660a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:10.844117 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acec7f7-8b24-42a0-beeb-bac23aa6660a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:10.850471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acec7f7-8b24-42a0-beeb-bac23aa6660a admin admin] PATH_INFO: `/v3/projects/fcf422f003424bb1954e85f63280424a` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:10.857199 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acec7f7-8b24-42a0-beeb-bac23aa6660a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcf422f003424bb1954e85f63280424a)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:10.871185 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acec7f7-8b24-42a0-beeb-bac23aa6660a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:11.005949 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:11.011333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1acec7f7-8b24-42a0-beeb-bac23aa6660a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcf422f003424bb1954e85f63280424a', 'request_id': 'req-1acec7f7-8b24-42a0-beeb-bac23aa6660a'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:11.061631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1acec7f7-8b24-42a0-beeb-bac23aa6660a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcf422f003424bb1954e85f63280424a', 'request_id': 'req-1acec7f7-8b24-42a0-beeb-bac23aa6660a'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:11.092454 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1acec7f7-8b24-42a0-beeb-bac23aa6660a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcf422f003424bb1954e85f63280424a', 'request_id': 'req-1acec7f7-8b24-42a0-beeb-bac23aa6660a'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:11.101682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f45696-88cb-4ba6-91d8-e195dc854ad0 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:11.103666 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d0f45696-88cb-4ba6-91d8-e195dc854ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:11.140668 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1acec7f7-8b24-42a0-beeb-bac23aa6660a admin admin] The token cache is being invalidate because project fcf422f003424bb1954e85f63280424a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:11.140904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1acec7f7-8b24-42a0-beeb-bac23aa6660a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:11.142730 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f45696-88cb-4ba6-91d8-e195dc854ad0 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:11.161221 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1acec7f7-8b24-42a0-beeb-bac23aa6660a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:11.167981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 893/1785] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:10 2022] DELETE /identity/v3/projects/fcf422f003424bb1954e85f63280424a => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:11.173021 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc7b56-62c2-451f-b385-a8a00e5feaf1 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:11.175115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc7b56-62c2-451f-b385-a8a00e5feaf1 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:11.176062 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc7b56-62c2-451f-b385-a8a00e5feaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:11.266032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-87fc7b56-62c2-451f-b385-a8a00e5feaf1 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:11.283188 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-87fc7b56-62c2-451f-b385-a8a00e5feaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:11.423697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9de86f-896f-4930-9835-cc194df78552 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '066f69a87f5f48c7b20c1e0fcf5c1107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:11.429034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9de86f-896f-4930-9835-cc194df78552 service placement] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:11.435225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9de86f-896f-4930-9835-cc194df78552 service placement] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:11.435818 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9de86f-896f-4930-9835-cc194df78552 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:11.445357 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-de9de86f-896f-4930-9835-cc194df78552 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:11.474659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9de86f-896f-4930-9835-cc194df78552 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:11.479092 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9de86f-896f-4930-9835-cc194df78552 service placement] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:11.565943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 893/1786] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:11.626634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 894/1787] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:11.629365 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa8bc9-8aa0-40ff-b8f4-a4b4b4509db8 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:11.629943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa8bc9-8aa0-40ff-b8f4-a4b4b4509db8 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:11.630384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa8bc9-8aa0-40ff-b8f4-a4b4b4509db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:11.671265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-affa8bc9-8aa0-40ff-b8f4-a4b4b4509db8 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:11.674827 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-affa8bc9-8aa0-40ff-b8f4-a4b4b4509db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:11.703678 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc7b56-62c2-451f-b385-a8a00e5feaf1 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:11.707965 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 894/1788] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:11.735099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc7b56-62c2-451f-b385-a8a00e5feaf1 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:11.746390 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ad4ae-4aae-4712-94bc-24db590f5e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:11.747356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ad4ae-4aae-4712-94bc-24db590f5e62 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:11.747773 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ad4ae-4aae-4712-94bc-24db590f5e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:11.748144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ad4ae-4aae-4712-94bc-24db590f5e62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:11.749121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ad4ae-4aae-4712-94bc-24db590f5e62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:11.749539 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ad4ae-4aae-4712-94bc-24db590f5e62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:11.751461 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ad4ae-4aae-4712-94bc-24db590f5e62 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:11.781755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 895/1789] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:13:11.786490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa8bc9-8aa0-40ff-b8f4-a4b4b4509db8 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:11.802044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa8bc9-8aa0-40ff-b8f4-a4b4b4509db8 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:11.815234 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa21279-ca2b-467f-8d35-325bc573cc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:11.816326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa21279-ca2b-467f-8d35-325bc573cc90 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:11.817101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa21279-ca2b-467f-8d35-325bc573cc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:11.818495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa21279-ca2b-467f-8d35-325bc573cc90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:11.819498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa21279-ca2b-467f-8d35-325bc573cc90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:11.827855 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa21279-ca2b-467f-8d35-325bc573cc90 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:11.916743 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 895/1790] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:11 2022] POST /identity/v3/projects => generated 369 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:11.920648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ad4ae-4aae-4712-94bc-24db590f5e62 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:11.934393 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ad4ae-4aae-4712-94bc-24db590f5e62 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:12.040581 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212676eb-7559-48e3-8418-836d526fb444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:12.041433 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212676eb-7559-48e3-8418-836d526fb444 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:12.041727 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212676eb-7559-48e3-8418-836d526fb444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:12.042012 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212676eb-7559-48e3-8418-836d526fb444 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:12.042747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212676eb-7559-48e3-8418-836d526fb444 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:12.048540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212676eb-7559-48e3-8418-836d526fb444 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:12.122291 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 896/1791] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:11 2022] POST /identity/v3/users => generated 285 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:12.125939 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa21279-ca2b-467f-8d35-325bc573cc90 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:12.141416 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa21279-ca2b-467f-8d35-325bc573cc90 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:12.159401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92543935-eabb-4a42-b2d1-12c6efb1b0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:12.160706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92543935-eabb-4a42-b2d1-12c6efb1b0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:12.161213 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92543935-eabb-4a42-b2d1-12c6efb1b0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:12.161755 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92543935-eabb-4a42-b2d1-12c6efb1b0e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:12.162844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92543935-eabb-4a42-b2d1-12c6efb1b0e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:12.164748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92543935-eabb-4a42-b2d1-12c6efb1b0e1 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:12.195043 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 896/1792] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:12.196734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212676eb-7559-48e3-8418-836d526fb444 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:12.209986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212676eb-7559-48e3-8418-836d526fb444 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:12.216568 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1882b95-f4c3-49dd-aa26-cf7e384d7fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:12.217585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1882b95-f4c3-49dd-aa26-cf7e384d7fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:12.218156 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1882b95-f4c3-49dd-aa26-cf7e384d7fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:12.219044 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1882b95-f4c3-49dd-aa26-cf7e384d7fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:12.220325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1882b95-f4c3-49dd-aa26-cf7e384d7fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:12.222573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1882b95-f4c3-49dd-aa26-cf7e384d7fd6 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:12.248450 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 897/1793] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:12.251155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92543935-eabb-4a42-b2d1-12c6efb1b0e1 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:12.267184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92543935-eabb-4a42-b2d1-12c6efb1b0e1 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:12.307580 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae703d8-dbf6-4926-b13a-53fb7cdb90b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:12.310693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae703d8-dbf6-4926-b13a-53fb7cdb90b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:12.311202 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae703d8-dbf6-4926-b13a-53fb7cdb90b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:12.313346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae703d8-dbf6-4926-b13a-53fb7cdb90b2 admin admin] PATH_INFO: `/v3/projects/9b9f5f9f643a447ab071ca5dd6210edc/users/de9f1f0e2a424a99b04186da6fb87bcf/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:12.352283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae703d8-dbf6-4926-b13a-53fb7cdb90b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b9f5f9f643a447ab071ca5dd6210edc, user_id=de9f1f0e2a424a99b04186da6fb87bcf, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:12.357028 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae703d8-dbf6-4926-b13a-53fb7cdb90b2 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:12.383145 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 897/1794] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:12 2022] PUT /identity/v3/projects/9b9f5f9f643a447ab071ca5dd6210edc/users/de9f1f0e2a424a99b04186da6fb87bcf/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:12.395312 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9fb95-811f-425a-be37-42f325c50049 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:12.396170 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9fb95-811f-425a-be37-42f325c50049 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:12.396625 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9fb95-811f-425a-be37-42f325c50049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:12.451377 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a6d9fb95-811f-425a-be37-42f325c50049 None None] MFA Rules not processed for user `de9f1f0e2a424a99b04186da6fb87bcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:12.452982 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a6d9fb95-811f-425a-be37-42f325c50049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:12.536533 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae703d8-dbf6-4926-b13a-53fb7cdb90b2 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:12.537326 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:12.543944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-dae703d8-dbf6-4926-b13a-53fb7cdb90b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:12.630278 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 898/1795] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:12.635046 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9fb95-811f-425a-be37-42f325c50049 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:12.900201 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae703d8-dbf6-4926-b13a-53fb7cdb90b2 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:12.905664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9fb95-811f-425a-be37-42f325c50049 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:12.914403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df883a-02d7-4705-aaca-853271b7441a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:12.916477 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df883a-02d7-4705-aaca-853271b7441a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:12.916745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df883a-02d7-4705-aaca-853271b7441a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:12.925974 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df883a-02d7-4705-aaca-853271b7441a admin admin] PATH_INFO: `/v3/users/8bfbbba3ae0b41aebaf6e44acada71f2` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:12.932144 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df883a-02d7-4705-aaca-853271b7441a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bfbbba3ae0b41aebaf6e44acada71f2)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:12.937138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df883a-02d7-4705-aaca-853271b7441a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:12.940777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a2927-a51c-48a1-9882-baec202bef8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:12.941633 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a2927-a51c-48a1-9882-baec202bef8a admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:12.943288 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a2927-a51c-48a1-9882-baec202bef8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:12.944363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a2927-a51c-48a1-9882-baec202bef8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:12.947456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a2927-a51c-48a1-9882-baec202bef8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:12.952566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a2927-a51c-48a1-9882-baec202bef8a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:12.964200 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:13.089325 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-f3df883a-02d7-4705-aaca-853271b7441a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bfbbba3ae0b41aebaf6e44acada71f2'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:13.118724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 899/1796] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:12 2022] POST /identity/v3/projects => generated 367 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:13.120616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a2927-a51c-48a1-9882-baec202bef8a admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:13.148833 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-f3df883a-02d7-4705-aaca-853271b7441a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bfbbba3ae0b41aebaf6e44acada71f2'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:13.168258 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-f3df883a-02d7-4705-aaca-853271b7441a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bfbbba3ae0b41aebaf6e44acada71f2'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:13.169443 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-f3df883a-02d7-4705-aaca-853271b7441a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bfbbba3ae0b41aebaf6e44acada71f2'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:13.191897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-f3df883a-02d7-4705-aaca-853271b7441a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bfbbba3ae0b41aebaf6e44acada71f2'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:13.214818 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 898/1797] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:12 2022] DELETE /identity/v3/users/8bfbbba3ae0b41aebaf6e44acada71f2 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:13.217681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00edd00-9200-4924-b8ff-2055f2cc3fe6 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:13.218981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00edd00-9200-4924-b8ff-2055f2cc3fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:13.219499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00edd00-9200-4924-b8ff-2055f2cc3fe6 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:13.223892 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 899/1798] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jul 20 09:13:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:13:13.229332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14538c1-2d85-4d53-b04c-5c791ad625c6 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:13.230950 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14538c1-2d85-4d53-b04c-5c791ad625c6 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:13.231363 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14538c1-2d85-4d53-b04c-5c791ad625c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:13.292156 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-e14538c1-2d85-4d53-b04c-5c791ad625c6 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:13.294967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e14538c1-2d85-4d53-b04c-5c791ad625c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:13.374514 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a2927-a51c-48a1-9882-baec202bef8a admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:13.495954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 900/1799] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:13.567350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f00b4-a7a2-4c68-bdea-eabf9ede906f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:13.569881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f00b4-a7a2-4c68-bdea-eabf9ede906f admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:13.570321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f00b4-a7a2-4c68-bdea-eabf9ede906f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:13.570770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f00b4-a7a2-4c68-bdea-eabf9ede906f admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:13.573708 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f00b4-a7a2-4c68-bdea-eabf9ede906f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:13.575940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f00b4-a7a2-4c68-bdea-eabf9ede906f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:13.589107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:13.611799 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 900/1800] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:13 2022] POST /identity/v3/users => generated 290 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:13.615536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14538c1-2d85-4d53-b04c-5c791ad625c6 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:13.619919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e14538c1-2d85-4d53-b04c-5c791ad625c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:13.646550 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14538c1-2d85-4d53-b04c-5c791ad625c6 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:13.652908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070a908-af7d-404b-a636-e937d60acef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:13.653867 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070a908-af7d-404b-a636-e937d60acef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:13.654295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070a908-af7d-404b-a636-e937d60acef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:13.654760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070a908-af7d-404b-a636-e937d60acef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:13.655642 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070a908-af7d-404b-a636-e937d60acef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:13.657467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070a908-af7d-404b-a636-e937d60acef7 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:13.674488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 901/1801] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:13.679141 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f00b4-a7a2-4c68-bdea-eabf9ede906f admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:13.701430 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f00b4-a7a2-4c68-bdea-eabf9ede906f admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:13.706919 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cb10f-e30e-45f6-9b83-b8c2a1bcf2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:13.708007 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cb10f-e30e-45f6-9b83-b8c2a1bcf2f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:13.708592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cb10f-e30e-45f6-9b83-b8c2a1bcf2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:13.709050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cb10f-e30e-45f6-9b83-b8c2a1bcf2f0 admin admin] PATH_INFO: `/v3/projects/f4549a677ce640ca96d33aeedd9abb40/users/79532b2f9a794e5581515a63a4665401/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:13.732020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57cb10f-e30e-45f6-9b83-b8c2a1bcf2f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4549a677ce640ca96d33aeedd9abb40, user_id=79532b2f9a794e5581515a63a4665401, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:13.738352 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c50cca-ef16-4a26-b9ba-87eca256be1a None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:13.738648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c50cca-ef16-4a26-b9ba-87eca256be1a None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:13.739071 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c50cca-ef16-4a26-b9ba-87eca256be1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:13.750485 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57cb10f-e30e-45f6-9b83-b8c2a1bcf2f0 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:13.786318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-30c50cca-ef16-4a26-b9ba-87eca256be1a None None] MFA Rules not processed for user `be08c43f0d0c4a8194596e54500b44ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:13.787639 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-30c50cca-ef16-4a26-b9ba-87eca256be1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:13.792088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 901/1802] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:13 2022] PUT /identity/v3/projects/f4549a677ce640ca96d33aeedd9abb40/users/79532b2f9a794e5581515a63a4665401/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:13.794278 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cb10f-e30e-45f6-9b83-b8c2a1bcf2f0 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:14.069649 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 902/1803] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jul 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:14.076162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c50cca-ef16-4a26-b9ba-87eca256be1a None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:14.077001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cb10f-e30e-45f6-9b83-b8c2a1bcf2f0 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:14.086329 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-30c50cca-ef16-4a26-b9ba-87eca256be1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:14.105784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855ae71-f409-4fde-bd10-fa5ccf688b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:14.106412 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855ae71-f409-4fde-bd10-fa5ccf688b04 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:14.106708 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855ae71-f409-4fde-bd10-fa5ccf688b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:14.107007 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855ae71-f409-4fde-bd10-fa5ccf688b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:14.111829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855ae71-f409-4fde-bd10-fa5ccf688b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:14.113540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855ae71-f409-4fde-bd10-fa5ccf688b04 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:14.152703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 902/1804] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:14.167950 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:14.168575 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855ae71-f409-4fde-bd10-fa5ccf688b04 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:14.204423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855ae71-f409-4fde-bd10-fa5ccf688b04 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:14.214975 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44910bf8-a723-400e-87d2-3e155e6d0c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:14.216132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44910bf8-a723-400e-87d2-3e155e6d0c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:14.216404 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44910bf8-a723-400e-87d2-3e155e6d0c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:14.216681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44910bf8-a723-400e-87d2-3e155e6d0c05 admin admin] PATH_INFO: `/v3/projects/f4549a677ce640ca96d33aeedd9abb40/users/79532b2f9a794e5581515a63a4665401/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:14.219740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44910bf8-a723-400e-87d2-3e155e6d0c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4549a677ce640ca96d33aeedd9abb40, user_id=79532b2f9a794e5581515a63a4665401, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:14.228116 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44910bf8-a723-400e-87d2-3e155e6d0c05 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:14.262760 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 903/1805] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:14 2022] PUT /identity/v3/projects/f4549a677ce640ca96d33aeedd9abb40/users/79532b2f9a794e5581515a63a4665401/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:14.274130 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c0af2-8bf0-47d7-888f-029ea439b874 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:14.274528 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c0af2-8bf0-47d7-888f-029ea439b874 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:14.274989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c0af2-8bf0-47d7-888f-029ea439b874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:14.301029 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c50cca-ef16-4a26-b9ba-87eca256be1a None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:14.319079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-5d8c0af2-8bf0-47d7-888f-029ea439b874 None None] MFA Rules not processed for user `79532b2f9a794e5581515a63a4665401`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:14.321018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5d8c0af2-8bf0-47d7-888f-029ea439b874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:14.544625 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c29c1-729a-44ad-976b-9e1490120fcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:14.550853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 904/1806] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:14 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:14.553436 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c29c1-729a-44ad-976b-9e1490120fcd service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:14.566590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c0af2-8bf0-47d7-888f-029ea439b874 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:14.568665 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5d8c0af2-8bf0-47d7-888f-029ea439b874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:14.574073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c29c1-729a-44ad-976b-9e1490120fcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:14.575567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c29c1-729a-44ad-976b-9e1490120fcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:14.695150 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:14.742146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c0af2-8bf0-47d7-888f-029ea439b874 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:14.752447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d58230-ec7b-464f-b2d9-05b6588bc6bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:14.753129 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d58230-ec7b-464f-b2d9-05b6588bc6bd service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:14.753412 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d58230-ec7b-464f-b2d9-05b6588bc6bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:14.753693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d58230-ec7b-464f-b2d9-05b6588bc6bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:14.792466 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d58230-ec7b-464f-b2d9-05b6588bc6bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:14.793963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d58230-ec7b-464f-b2d9-05b6588bc6bd service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:14.817377 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 905/1807] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:14 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:14.870210 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c29c1-729a-44ad-976b-9e1490120fcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:14.893540 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c29c1-729a-44ad-976b-9e1490120fcd service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:14.932547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 903/1808] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 862 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:15.472986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06a25f-58b7-4872-a709-99af14737deb None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:15.473280 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06a25f-58b7-4872-a709-99af14737deb None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:15.473565 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06a25f-58b7-4872-a709-99af14737deb None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:15.475272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 906/1809] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jul 20 09:13:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:13:15.572226 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af8173-333a-4668-9c1f-fcbf51104205 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:15.572480 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af8173-333a-4668-9c1f-fcbf51104205 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:15.572764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af8173-333a-4668-9c1f-fcbf51104205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:15.627774 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-89af8173-333a-4668-9c1f-fcbf51104205 None None] MFA Rules not processed for user `be08c43f0d0c4a8194596e54500b44ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:15.629210 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-89af8173-333a-4668-9c1f-fcbf51104205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:15.801949 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06a25f-58b7-4872-a709-99af14737deb None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:15.803803 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 904/1810] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jul 20 09:13:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:15.806875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5c06a25f-58b7-4872-a709-99af14737deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:15.846928 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06a25f-58b7-4872-a709-99af14737deb None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:15.857233 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50031c-26af-4b4c-81dc-4e4778819fff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:15.857881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50031c-26af-4b4c-81dc-4e4778819fff service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:15.858177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50031c-26af-4b4c-81dc-4e4778819fff service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:15.858463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50031c-26af-4b4c-81dc-4e4778819fff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:15.890961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50031c-26af-4b4c-81dc-4e4778819fff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:15.892682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50031c-26af-4b4c-81dc-4e4778819fff service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:15.918764 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 907/1811] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:15.928613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a1621-98d4-45a7-b88a-f77998c20018 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:15.930020 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a1621-98d4-45a7-b88a-f77998c20018 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:15.930511 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a1621-98d4-45a7-b88a-f77998c20018 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:15.932879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 905/1812] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jul 20 09:13:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:13:15.982784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271852d3-a270-4830-9029-c01b88111e4e None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:15.983111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271852d3-a270-4830-9029-c01b88111e4e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:15.983387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271852d3-a270-4830-9029-c01b88111e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:16.034246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-271852d3-a270-4830-9029-c01b88111e4e None None] MFA Rules not processed for user `066f69a87f5f48c7b20c1e0fcf5c1107`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:16.035961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-271852d3-a270-4830-9029-c01b88111e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:16.245453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 908/1813] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jul 20 09:13:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:16.252609 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a1621-98d4-45a7-b88a-f77998c20018 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:16.254766 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-116a1621-98d4-45a7-b88a-f77998c20018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:16.286953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a1621-98d4-45a7-b88a-f77998c20018 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:16.300154 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67655311-d01b-40cc-8721-9d8935b88b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:16.301044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67655311-d01b-40cc-8721-9d8935b88b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:16.301333 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67655311-d01b-40cc-8721-9d8935b88b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:16.301614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67655311-d01b-40cc-8721-9d8935b88b7e admin admin] PATH_INFO: `/v3/users/7b698809b84e4b2082b9d4f5a03ef500` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:16.310252 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271852d3-a270-4830-9029-c01b88111e4e None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:16.311713 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-271852d3-a270-4830-9029-c01b88111e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:16.316549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67655311-d01b-40cc-8721-9d8935b88b7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b698809b84e4b2082b9d4f5a03ef500)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:16.329069 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67655311-d01b-40cc-8721-9d8935b88b7e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:16.339739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271852d3-a270-4830-9029-c01b88111e4e None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:16.353852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:16.363432 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d9999-9fc3-4961-a66e-771e0e221164 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '066f69a87f5f48c7b20c1e0fcf5c1107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:16.365001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d9999-9fc3-4961-a66e-771e0e221164 service placement] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:16.366285 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d9999-9fc3-4961-a66e-771e0e221164 service placement] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:16.368885 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d9999-9fc3-4961-a66e-771e0e221164 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:16.392859 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d9999-9fc3-4961-a66e-771e0e221164 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:16.394869 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d9999-9fc3-4961-a66e-771e0e221164 service placement] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:16.434868 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 909/1814] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:16.469110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-67655311-d01b-40cc-8721-9d8935b88b7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b698809b84e4b2082b9d4f5a03ef500'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:16.494094 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-67655311-d01b-40cc-8721-9d8935b88b7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b698809b84e4b2082b9d4f5a03ef500'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:16.547721 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-67655311-d01b-40cc-8721-9d8935b88b7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b698809b84e4b2082b9d4f5a03ef500'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:16.550614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-67655311-d01b-40cc-8721-9d8935b88b7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b698809b84e4b2082b9d4f5a03ef500'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:16.574986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-67655311-d01b-40cc-8721-9d8935b88b7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b698809b84e4b2082b9d4f5a03ef500'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:16.589652 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 906/1815] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:16 2022] DELETE /identity/v3/users/7b698809b84e4b2082b9d4f5a03ef500 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:16.692063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d9999-9fc3-4961-a66e-771e0e221164 service placement] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:16.720733 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d9999-9fc3-4961-a66e-771e0e221164 service placement] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:16.888428 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4525da-aba2-4d28-872e-b08922e9fc10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:16.890181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4525da-aba2-4d28-872e-b08922e9fc10 service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:16.892038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4525da-aba2-4d28-872e-b08922e9fc10 service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:16.892875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4525da-aba2-4d28-872e-b08922e9fc10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:16.896899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ad4525da-aba2-4d28-872e-b08922e9fc10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:16.936900 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4525da-aba2-4d28-872e-b08922e9fc10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:16.938531 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4525da-aba2-4d28-872e-b08922e9fc10 service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:16.980337 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 910/1816] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:17.372694 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67655311-d01b-40cc-8721-9d8935b88b7e admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:17.375044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-67655311-d01b-40cc-8721-9d8935b88b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:17.534493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67655311-d01b-40cc-8721-9d8935b88b7e admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:17.581784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53153ca4-548a-4570-961c-c0e9f51f300d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:17.582936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53153ca4-548a-4570-961c-c0e9f51f300d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:17.583324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53153ca4-548a-4570-961c-c0e9f51f300d admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:17.583777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53153ca4-548a-4570-961c-c0e9f51f300d admin admin] PATH_INFO: `/v3/projects/90c3385d787e48f1b24a9413a458ff57` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:17.586211 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53153ca4-548a-4570-961c-c0e9f51f300d admin admin] RBAC: Authorizing `identity:delete_project(project_id=90c3385d787e48f1b24a9413a458ff57)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:17.589370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53153ca4-548a-4570-961c-c0e9f51f300d admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:17.593584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4525da-aba2-4d28-872e-b08922e9fc10 service nova] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:17.599413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ad4525da-aba2-4d28-872e-b08922e9fc10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:17.633137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4525da-aba2-4d28-872e-b08922e9fc10 service nova] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:17.663899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce28a3-39a7-432d-bee4-d39a89aef163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:17.669607 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce28a3-39a7-432d-bee4-d39a89aef163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:17.670190 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce28a3-39a7-432d-bee4-d39a89aef163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:17.670835 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce28a3-39a7-432d-bee4-d39a89aef163 admin admin] PATH_INFO: `/v3/projects/38111352942d40be8a50621adc1cff4f` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:17.689205 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce28a3-39a7-432d-bee4-d39a89aef163 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38111352942d40be8a50621adc1cff4f)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:17.702396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce28a3-39a7-432d-bee4-d39a89aef163 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:17.704893 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-53153ca4-548a-4570-961c-c0e9f51f300d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90c3385d787e48f1b24a9413a458ff57', 'request_id': 'req-53153ca4-548a-4570-961c-c0e9f51f300d'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:17.714480 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-53153ca4-548a-4570-961c-c0e9f51f300d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90c3385d787e48f1b24a9413a458ff57', 'request_id': 'req-53153ca4-548a-4570-961c-c0e9f51f300d'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:17.731879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-53153ca4-548a-4570-961c-c0e9f51f300d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90c3385d787e48f1b24a9413a458ff57', 'request_id': 'req-53153ca4-548a-4570-961c-c0e9f51f300d'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:17.764794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-53153ca4-548a-4570-961c-c0e9f51f300d admin admin] The token cache is being invalidate because project 90c3385d787e48f1b24a9413a458ff57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:17.765178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-53153ca4-548a-4570-961c-c0e9f51f300d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:17.767824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-53153ca4-548a-4570-961c-c0e9f51f300d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:17.780795 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 907/1817] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:17 2022] DELETE /identity/v3/projects/90c3385d787e48f1b24a9413a458ff57 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:17.783530 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53153ca4-548a-4570-961c-c0e9f51f300d admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:17.788437 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-53153ca4-548a-4570-961c-c0e9f51f300d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:17.853685 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:17.887090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:17.917750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b9ce28a3-39a7-432d-bee4-d39a89aef163 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38111352942d40be8a50621adc1cff4f', 'request_id': 'req-b9ce28a3-39a7-432d-bee4-d39a89aef163'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:17.961178 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b9ce28a3-39a7-432d-bee4-d39a89aef163 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38111352942d40be8a50621adc1cff4f', 'request_id': 'req-b9ce28a3-39a7-432d-bee4-d39a89aef163'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:17.974173 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53153ca4-548a-4570-961c-c0e9f51f300d admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:17.988583 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b9ce28a3-39a7-432d-bee4-d39a89aef163 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38111352942d40be8a50621adc1cff4f', 'request_id': 'req-b9ce28a3-39a7-432d-bee4-d39a89aef163'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.026976 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b9ce28a3-39a7-432d-bee4-d39a89aef163 admin admin] The token cache is being invalidate because project 38111352942d40be8a50621adc1cff4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:18.027316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b9ce28a3-39a7-432d-bee4-d39a89aef163 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.028212 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b9ce28a3-39a7-432d-bee4-d39a89aef163 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.030423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 911/1818] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:17 2022] DELETE /identity/v3/projects/38111352942d40be8a50621adc1cff4f => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:18.034399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce28a3-39a7-432d-bee4-d39a89aef163 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:18.035965 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b9ce28a3-39a7-432d-bee4-d39a89aef163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:18.149807 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffbda5-5555-44c8-ae02-30be645cb94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:18.150756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffbda5-5555-44c8-ae02-30be645cb94a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:18.151059 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffbda5-5555-44c8-ae02-30be645cb94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:18.151332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffbda5-5555-44c8-ae02-30be645cb94a admin admin] PATH_INFO: `/v3/users/e2760b33d89f48a996f3f93cfc9acf91` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:18.155161 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffbda5-5555-44c8-ae02-30be645cb94a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2760b33d89f48a996f3f93cfc9acf91)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:18.157122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffbda5-5555-44c8-ae02-30be645cb94a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:18.228679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce28a3-39a7-432d-bee4-d39a89aef163 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:18.238925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ba254-98ad-4bf3-a4af-b17759b19c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:18.244912 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ba254-98ad-4bf3-a4af-b17759b19c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:18.247074 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ba254-98ad-4bf3-a4af-b17759b19c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:18.247547 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ba254-98ad-4bf3-a4af-b17759b19c51 admin admin] PATH_INFO: `/v3/users/c6f5518ec7314a5eabcb8655993a42ef` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:18.251401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ba254-98ad-4bf3-a4af-b17759b19c51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6f5518ec7314a5eabcb8655993a42ef)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:18.253198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-02ffbda5-5555-44c8-ae02-30be645cb94a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2760b33d89f48a996f3f93cfc9acf91'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.254430 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5ba254-98ad-4bf3-a4af-b17759b19c51 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:18.293378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-02ffbda5-5555-44c8-ae02-30be645cb94a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2760b33d89f48a996f3f93cfc9acf91'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.319916 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-02ffbda5-5555-44c8-ae02-30be645cb94a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2760b33d89f48a996f3f93cfc9acf91'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.322082 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-02ffbda5-5555-44c8-ae02-30be645cb94a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2760b33d89f48a996f3f93cfc9acf91'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.336382 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6b5ba254-98ad-4bf3-a4af-b17759b19c51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6f5518ec7314a5eabcb8655993a42ef'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.338792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-02ffbda5-5555-44c8-ae02-30be645cb94a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2760b33d89f48a996f3f93cfc9acf91'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.352165 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 908/1819] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:17 2022] DELETE /identity/v3/users/e2760b33d89f48a996f3f93cfc9acf91 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:18.353779 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffbda5-5555-44c8-ae02-30be645cb94a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:18.355461 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-02ffbda5-5555-44c8-ae02-30be645cb94a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:18.382269 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6b5ba254-98ad-4bf3-a4af-b17759b19c51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6f5518ec7314a5eabcb8655993a42ef'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.391789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6b5ba254-98ad-4bf3-a4af-b17759b19c51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6f5518ec7314a5eabcb8655993a42ef'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.392586 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6b5ba254-98ad-4bf3-a4af-b17759b19c51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6f5518ec7314a5eabcb8655993a42ef'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.409122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6b5ba254-98ad-4bf3-a4af-b17759b19c51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6f5518ec7314a5eabcb8655993a42ef'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.420634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 912/1820] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:18 2022] DELETE /identity/v3/users/c6f5518ec7314a5eabcb8655993a42ef => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:18.430405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:18.464499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffbda5-5555-44c8-ae02-30be645cb94a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:18.560792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df83a1-bc94-4c24-be9b-7bed2c40bbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:18.561704 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df83a1-bc94-4c24-be9b-7bed2c40bbb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:18.561981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df83a1-bc94-4c24-be9b-7bed2c40bbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:18.562253 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df83a1-bc94-4c24-be9b-7bed2c40bbb9 admin admin] PATH_INFO: `/v3/users/de9f1f0e2a424a99b04186da6fb87bcf` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:18.564113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df83a1-bc94-4c24-be9b-7bed2c40bbb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de9f1f0e2a424a99b04186da6fb87bcf)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:18.566173 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df83a1-bc94-4c24-be9b-7bed2c40bbb9 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:18.627386 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-88df83a1-bc94-4c24-be9b-7bed2c40bbb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de9f1f0e2a424a99b04186da6fb87bcf'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.645313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-88df83a1-bc94-4c24-be9b-7bed2c40bbb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de9f1f0e2a424a99b04186da6fb87bcf'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.653035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-88df83a1-bc94-4c24-be9b-7bed2c40bbb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de9f1f0e2a424a99b04186da6fb87bcf'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.653816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-88df83a1-bc94-4c24-be9b-7bed2c40bbb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de9f1f0e2a424a99b04186da6fb87bcf'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.665998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-88df83a1-bc94-4c24-be9b-7bed2c40bbb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de9f1f0e2a424a99b04186da6fb87bcf'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.674263 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 909/1821] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:18 2022] DELETE /identity/v3/users/de9f1f0e2a424a99b04186da6fb87bcf => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:18.678857 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b0705-5a81-4648-8ace-9556b373254e None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:18.679155 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b0705-5a81-4648-8ace-9556b373254e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:18.679441 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b0705-5a81-4648-8ace-9556b373254e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:18.700551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-0d0b0705-5a81-4648-8ace-9556b373254e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:18.701829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0d0b0705-5a81-4648-8ace-9556b373254e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:18.736707 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:18.772111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 913/1822] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:18.782271 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df83a1-bc94-4c24-be9b-7bed2c40bbb9 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:18.784089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-88df83a1-bc94-4c24-be9b-7bed2c40bbb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:18.809436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b0705-5a81-4648-8ace-9556b373254e None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:18.810945 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0d0b0705-5a81-4648-8ace-9556b373254e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:18.824724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b0705-5a81-4648-8ace-9556b373254e None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:18.832425 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ef177-37e9-4f78-899f-d24d5d1eaeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:18.833369 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ef177-37e9-4f78-899f-d24d5d1eaeed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:18.833741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ef177-37e9-4f78-899f-d24d5d1eaeed admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:18.834102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ef177-37e9-4f78-899f-d24d5d1eaeed admin admin] PATH_INFO: `/v3/projects/4779de95d327453c83e06ab6345411de` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:18.840871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ef177-37e9-4f78-899f-d24d5d1eaeed admin admin] RBAC: Authorizing `identity:delete_project(project_id=4779de95d327453c83e06ab6345411de)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:18.847317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ef177-37e9-4f78-899f-d24d5d1eaeed admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:18.866017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df83a1-bc94-4c24-be9b-7bed2c40bbb9 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:18.870391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2151ac-cf79-4bc5-bf6b-5f8adb054490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:18.871087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2151ac-cf79-4bc5-bf6b-5f8adb054490 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:18.871506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2151ac-cf79-4bc5-bf6b-5f8adb054490 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:18.871827 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2151ac-cf79-4bc5-bf6b-5f8adb054490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:18.886474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2151ac-cf79-4bc5-bf6b-5f8adb054490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:18.888078 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2151ac-cf79-4bc5-bf6b-5f8adb054490 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:18.912770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cf9ef177-37e9-4f78-899f-d24d5d1eaeed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4779de95d327453c83e06ab6345411de', 'request_id': 'req-cf9ef177-37e9-4f78-899f-d24d5d1eaeed'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.914163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 910/1823] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:18.915385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2151ac-cf79-4bc5-bf6b-5f8adb054490 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:18.917164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-da2151ac-cf79-4bc5-bf6b-5f8adb054490 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:18.920515 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cf9ef177-37e9-4f78-899f-d24d5d1eaeed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4779de95d327453c83e06ab6345411de', 'request_id': 'req-cf9ef177-37e9-4f78-899f-d24d5d1eaeed'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.931207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cf9ef177-37e9-4f78-899f-d24d5d1eaeed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4779de95d327453c83e06ab6345411de', 'request_id': 'req-cf9ef177-37e9-4f78-899f-d24d5d1eaeed'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.938079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2151ac-cf79-4bc5-bf6b-5f8adb054490 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:18.947563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cf9ef177-37e9-4f78-899f-d24d5d1eaeed admin admin] The token cache is being invalidate because project 4779de95d327453c83e06ab6345411de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:18.947860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cf9ef177-37e9-4f78-899f-d24d5d1eaeed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.948560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:18.948817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cf9ef177-37e9-4f78-899f-d24d5d1eaeed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:18.950641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 914/1824] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:18 2022] DELETE /identity/v3/projects/4779de95d327453c83e06ab6345411de => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:18.955359 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c07561-b00c-488e-a7db-59be91ee90c1 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:18.955856 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c07561-b00c-488e-a7db-59be91ee90c1 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:18.956140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c07561-b00c-488e-a7db-59be91ee90c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:18.976286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-67c07561-b00c-488e-a7db-59be91ee90c1 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:18.977796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-67c07561-b00c-488e-a7db-59be91ee90c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:19.024957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d108c55-3378-4aa4-b44a-cde013068d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:19.026493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d108c55-3378-4aa4-b44a-cde013068d66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:19.026615 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d108c55-3378-4aa4-b44a-cde013068d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:19.027053 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d108c55-3378-4aa4-b44a-cde013068d66 admin admin] PATH_INFO: `/v3/projects/c8fa54f78c5341e69c04f3f61cdb5eba` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:19.028906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d108c55-3378-4aa4-b44a-cde013068d66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8fa54f78c5341e69c04f3f61cdb5eba)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:19.030936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d108c55-3378-4aa4-b44a-cde013068d66 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:19.050719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 915/1825] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:19.054871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd034bd8-bf60-4f7e-a4bb-0bd16d956f2e None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:19.055329 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd034bd8-bf60-4f7e-a4bb-0bd16d956f2e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:19.056229 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd034bd8-bf60-4f7e-a4bb-0bd16d956f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:19.077728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-cd034bd8-bf60-4f7e-a4bb-0bd16d956f2e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:19.079141 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-cd034bd8-bf60-4f7e-a4bb-0bd16d956f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:19.094780 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 916/1826] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:19.095884 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd034bd8-bf60-4f7e-a4bb-0bd16d956f2e None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:19.105769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd034bd8-bf60-4f7e-a4bb-0bd16d956f2e None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:19.110184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2d108c55-3378-4aa4-b44a-cde013068d66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8fa54f78c5341e69c04f3f61cdb5eba', 'request_id': 'req-2d108c55-3378-4aa4-b44a-cde013068d66'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:19.117031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2d108c55-3378-4aa4-b44a-cde013068d66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8fa54f78c5341e69c04f3f61cdb5eba', 'request_id': 'req-2d108c55-3378-4aa4-b44a-cde013068d66'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:19.125726 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2d108c55-3378-4aa4-b44a-cde013068d66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8fa54f78c5341e69c04f3f61cdb5eba', 'request_id': 'req-2d108c55-3378-4aa4-b44a-cde013068d66'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:19.148373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2d108c55-3378-4aa4-b44a-cde013068d66 admin admin] The token cache is being invalidate because project c8fa54f78c5341e69c04f3f61cdb5eba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:19.148717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2d108c55-3378-4aa4-b44a-cde013068d66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:19.149455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2d108c55-3378-4aa4-b44a-cde013068d66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:19.151230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 911/1827] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:18 2022] DELETE /identity/v3/projects/c8fa54f78c5341e69c04f3f61cdb5eba => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:19.156560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1a594-268e-43e9-b777-4d08bd96d0ac None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:19.156917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1a594-268e-43e9-b777-4d08bd96d0ac None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:19.157209 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1a594-268e-43e9-b777-4d08bd96d0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:19.184161 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-f5f1a594-268e-43e9-b777-4d08bd96d0ac None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:19.185440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f5f1a594-268e-43e9-b777-4d08bd96d0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:19.197856 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:19.208820 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14636762-9f0a-4f3c-8007-e27376f4fdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:19.209964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14636762-9f0a-4f3c-8007-e27376f4fdef admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:19.210735 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14636762-9f0a-4f3c-8007-e27376f4fdef admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:19.211322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14636762-9f0a-4f3c-8007-e27376f4fdef admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:19.213174 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14636762-9f0a-4f3c-8007-e27376f4fdef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:19.214018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14636762-9f0a-4f3c-8007-e27376f4fdef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:19.217554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14636762-9f0a-4f3c-8007-e27376f4fdef admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:19.241059 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 917/1828] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:13:19.242107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14636762-9f0a-4f3c-8007-e27376f4fdef admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:19.245160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-14636762-9f0a-4f3c-8007-e27376f4fdef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:19.264826 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14636762-9f0a-4f3c-8007-e27376f4fdef admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:19.272021 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe74b1c-7bfc-40fb-99dc-73b59bd0647c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:19.273134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe74b1c-7bfc-40fb-99dc-73b59bd0647c admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:19.273508 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe74b1c-7bfc-40fb-99dc-73b59bd0647c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:19.273792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe74b1c-7bfc-40fb-99dc-73b59bd0647c admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:19.276726 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe74b1c-7bfc-40fb-99dc-73b59bd0647c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:19.277524 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe74b1c-7bfc-40fb-99dc-73b59bd0647c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:19.290876 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 912/1829] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:19.292740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f9a36-feec-46be-8ad7-b7077b6a7302 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:19.293055 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f9a36-feec-46be-8ad7-b7077b6a7302 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:19.293338 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f9a36-feec-46be-8ad7-b7077b6a7302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:19.331994 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-ed3f9a36-feec-46be-8ad7-b7077b6a7302 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:19.332198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 918/1830] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:19 2022] POST /identity/v3/projects => generated 363 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:13:19.333305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe74b1c-7bfc-40fb-99dc-73b59bd0647c admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:19.333886 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ed3f9a36-feec-46be-8ad7-b7077b6a7302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:19.399358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:19.437246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 913/1831] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:19.441446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f9a36-feec-46be-8ad7-b7077b6a7302 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:19.479910 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f9a36-feec-46be-8ad7-b7077b6a7302 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:19.496628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17760369-6490-45e5-97e0-12826d51438c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:19.496628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17760369-6490-45e5-97e0-12826d51438c admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:19.496628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17760369-6490-45e5-97e0-12826d51438c admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:19.496628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17760369-6490-45e5-97e0-12826d51438c admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:19.497630 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17760369-6490-45e5-97e0-12826d51438c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:19.497954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17760369-6490-45e5-97e0-12826d51438c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:19.499726 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17760369-6490-45e5-97e0-12826d51438c admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:19.505196 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe74b1c-7bfc-40fb-99dc-73b59bd0647c admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:19.514262 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a203d-ebf6-4600-b5b3-644e73bcb46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:19.514981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a203d-ebf6-4600-b5b3-644e73bcb46c admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:19.515259 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a203d-ebf6-4600-b5b3-644e73bcb46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:19.515533 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a203d-ebf6-4600-b5b3-644e73bcb46c admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:19.518419 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a203d-ebf6-4600-b5b3-644e73bcb46c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:19.518419 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a203d-ebf6-4600-b5b3-644e73bcb46c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:19.534814 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 914/1832] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:13:19.535930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17760369-6490-45e5-97e0-12826d51438c admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:19.538173 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-17760369-6490-45e5-97e0-12826d51438c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:19.574511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 919/1833] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:19 2022] POST /identity/v3/users => generated 282 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:19.574511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a203d-ebf6-4600-b5b3-644e73bcb46c admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:19.592542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a203d-ebf6-4600-b5b3-644e73bcb46c admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:19.601023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17760369-6490-45e5-97e0-12826d51438c admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:19.607062 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03594c0f-f989-427e-aafc-f1c5b884c60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:19.607729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03594c0f-f989-427e-aafc-f1c5b884c60e admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:19.608013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03594c0f-f989-427e-aafc-f1c5b884c60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:19.608283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03594c0f-f989-427e-aafc-f1c5b884c60e admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:19.608989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03594c0f-f989-427e-aafc-f1c5b884c60e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:19.610906 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03594c0f-f989-427e-aafc-f1c5b884c60e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:19.614127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec371b0c-f920-473d-af65-d96668f38917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:19.615035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec371b0c-f920-473d-af65-d96668f38917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:19.615316 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec371b0c-f920-473d-af65-d96668f38917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:19.615591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec371b0c-f920-473d-af65-d96668f38917 admin admin] PATH_INFO: `/v3/projects/9b9f5f9f643a447ab071ca5dd6210edc` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:19.618606 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec371b0c-f920-473d-af65-d96668f38917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b9f5f9f643a447ab071ca5dd6210edc)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:19.621300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec371b0c-f920-473d-af65-d96668f38917 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:19.674999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 920/1834] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:19 2022] POST /identity/v3/projects => generated 391 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:19.674999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03594c0f-f989-427e-aafc-f1c5b884c60e admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:19.718111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-ec371b0c-f920-473d-af65-d96668f38917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b9f5f9f643a447ab071ca5dd6210edc', 'request_id': 'req-ec371b0c-f920-473d-af65-d96668f38917'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:19.726063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-ec371b0c-f920-473d-af65-d96668f38917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b9f5f9f643a447ab071ca5dd6210edc', 'request_id': 'req-ec371b0c-f920-473d-af65-d96668f38917'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:19.756856 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-ec371b0c-f920-473d-af65-d96668f38917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b9f5f9f643a447ab071ca5dd6210edc', 'request_id': 'req-ec371b0c-f920-473d-af65-d96668f38917'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:19.782849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-ec371b0c-f920-473d-af65-d96668f38917 admin admin] The token cache is being invalidate because project 9b9f5f9f643a447ab071ca5dd6210edc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:19.783153 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-ec371b0c-f920-473d-af65-d96668f38917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:19.783815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-ec371b0c-f920-473d-af65-d96668f38917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:19.785541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 915/1835] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:19 2022] DELETE /identity/v3/projects/9b9f5f9f643a447ab071ca5dd6210edc => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:19.786279 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec371b0c-f920-473d-af65-d96668f38917 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:19.789476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ec371b0c-f920-473d-af65-d96668f38917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:19.802047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03594c0f-f989-427e-aafc-f1c5b884c60e admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:19.893009 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:19.904506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec371b0c-f920-473d-af65-d96668f38917 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:19.911428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77b9e9-e98e-4ee6-8d05-b46246297eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:19.912429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77b9e9-e98e-4ee6-8d05-b46246297eec admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:19.912560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77b9e9-e98e-4ee6-8d05-b46246297eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:19.912952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77b9e9-e98e-4ee6-8d05-b46246297eec admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:19.913721 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77b9e9-e98e-4ee6-8d05-b46246297eec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:19.915960 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77b9e9-e98e-4ee6-8d05-b46246297eec admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:19.919003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8514ff2-dbb3-4e94-9643-73fd6dfa0119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:19.919582 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8514ff2-dbb3-4e94-9643-73fd6dfa0119 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:19.919869 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8514ff2-dbb3-4e94-9643-73fd6dfa0119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:19.920295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8514ff2-dbb3-4e94-9643-73fd6dfa0119 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:19.921036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8514ff2-dbb3-4e94-9643-73fd6dfa0119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:19.923992 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8514ff2-dbb3-4e94-9643-73fd6dfa0119 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:19.944512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 921/1836] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:19.945507 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8514ff2-dbb3-4e94-9643-73fd6dfa0119 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:19.948651 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 916/1837] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:19 2022] POST /identity/v3/users => generated 296 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:19.949581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77b9e9-e98e-4ee6-8d05-b46246297eec admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:19.949990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b8514ff2-dbb3-4e94-9643-73fd6dfa0119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:19.951397 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6b77b9e9-e98e-4ee6-8d05-b46246297eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:19.963677 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77b9e9-e98e-4ee6-8d05-b46246297eec admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:19.964564 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8514ff2-dbb3-4e94-9643-73fd6dfa0119 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:19.969117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a36564-2a33-475b-85c0-31047f0ec77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:19.969955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a36564-2a33-475b-85c0-31047f0ec77f admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:19.970294 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a36564-2a33-475b-85c0-31047f0ec77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:19.970763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a36564-2a33-475b-85c0-31047f0ec77f admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:19.971396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4c54f-7475-4848-b021-76eb8210e25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:19.971527 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a36564-2a33-475b-85c0-31047f0ec77f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:19.972400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4c54f-7475-4848-b021-76eb8210e25b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:19.972749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4c54f-7475-4848-b021-76eb8210e25b admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:19.973108 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4c54f-7475-4848-b021-76eb8210e25b admin admin] PATH_INFO: `/v3/users/79532b2f9a794e5581515a63a4665401` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:19.973399 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a36564-2a33-475b-85c0-31047f0ec77f admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:19.974436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4c54f-7475-4848-b021-76eb8210e25b admin admin] RBAC: Authorizing `identity:delete_user(user_id=79532b2f9a794e5581515a63a4665401)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:19.976480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4c54f-7475-4848-b021-76eb8210e25b admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:19.985990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 917/1838] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:19.986878 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a36564-2a33-475b-85c0-31047f0ec77f admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:19.997757 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a36564-2a33-475b-85c0-31047f0ec77f admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:20.002455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805d8af-fb6b-4343-9752-55765a1f335a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:20.003157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805d8af-fb6b-4343-9752-55765a1f335a admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.003445 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805d8af-fb6b-4343-9752-55765a1f335a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.003724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805d8af-fb6b-4343-9752-55765a1f335a admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.004632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805d8af-fb6b-4343-9752-55765a1f335a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:20.006209 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805d8af-fb6b-4343-9752-55765a1f335a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:20.021272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 918/1839] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:20.021272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805d8af-fb6b-4343-9752-55765a1f335a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:20.029452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805d8af-fb6b-4343-9752-55765a1f335a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:20.034184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38b738-3636-45eb-8dc0-fb3db146d63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:20.035086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38b738-3636-45eb-8dc0-fb3db146d63a admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.035369 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38b738-3636-45eb-8dc0-fb3db146d63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.035652 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38b738-3636-45eb-8dc0-fb3db146d63a admin admin] PATH_INFO: `/v3/projects/1172dfc41b9041d2b884c03390cd372a/users/3e970d8238614224b0b7f82801ce5382/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.042195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-68a4c54f-7475-4848-b021-76eb8210e25b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79532b2f9a794e5581515a63a4665401'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:20.052115 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38b738-3636-45eb-8dc0-fb3db146d63a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1172dfc41b9041d2b884c03390cd372a, user_id=3e970d8238614224b0b7f82801ce5382, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:20.054480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-68a4c54f-7475-4848-b021-76eb8210e25b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79532b2f9a794e5581515a63a4665401'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:20.054861 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38b738-3636-45eb-8dc0-fb3db146d63a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:20.058665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:20.062616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-68a4c54f-7475-4848-b021-76eb8210e25b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79532b2f9a794e5581515a63a4665401'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:20.063432 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-68a4c54f-7475-4848-b021-76eb8210e25b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79532b2f9a794e5581515a63a4665401'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:20.071037 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 919/1840] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:20 2022] PUT /identity/v3/projects/1172dfc41b9041d2b884c03390cd372a/users/3e970d8238614224b0b7f82801ce5382/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:20.071967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38b738-3636-45eb-8dc0-fb3db146d63a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:20.072085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-68a4c54f-7475-4848-b021-76eb8210e25b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79532b2f9a794e5581515a63a4665401'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:20.076472 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ba38b738-3636-45eb-8dc0-fb3db146d63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:20.082715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 922/1841] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:19 2022] DELETE /identity/v3/users/79532b2f9a794e5581515a63a4665401 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:20.082715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de8d91-9e47-46dd-9a6c-9bd070a2904b None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.083043 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de8d91-9e47-46dd-9a6c-9bd070a2904b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.083370 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de8d91-9e47-46dd-9a6c-9bd070a2904b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.111193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a9de8d91-9e47-46dd-9a6c-9bd070a2904b None None] MFA Rules not processed for user `3e970d8238614224b0b7f82801ce5382`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:20.112409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a9de8d91-9e47-46dd-9a6c-9bd070a2904b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:20.165906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38b738-3636-45eb-8dc0-fb3db146d63a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:20.194893 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 923/1842] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:20 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:20.197932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de8d91-9e47-46dd-9a6c-9bd070a2904b None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:20.200675 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a9de8d91-9e47-46dd-9a6c-9bd070a2904b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:20.230137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d6a72-f172-490b-984e-218738570d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:20.232154 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d6a72-f172-490b-984e-218738570d3a admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.232744 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d6a72-f172-490b-984e-218738570d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.233607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d6a72-f172-490b-984e-218738570d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.234845 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d6a72-f172-490b-984e-218738570d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:20.237553 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d6a72-f172-490b-984e-218738570d3a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:20.251210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:20.256536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 920/1843] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:20.262515 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d6a72-f172-490b-984e-218738570d3a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:20.272942 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d6a72-f172-490b-984e-218738570d3a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:20.278925 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae662a4-fb77-444e-9b1b-bbbabe94aa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:20.280214 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae662a4-fb77-444e-9b1b-bbbabe94aa0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.280214 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae662a4-fb77-444e-9b1b-bbbabe94aa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.280429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae662a4-fb77-444e-9b1b-bbbabe94aa0e admin admin] PATH_INFO: `/v3/projects/3bcdbcfc080f4b09afb2ba8985e0bf86/users/92017633cf094918bbdad11d0a1f79a5/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.282020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de8d91-9e47-46dd-9a6c-9bd070a2904b None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:20.286799 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0938b-83ae-44a8-82ac-21d1b8d1a505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:20.287492 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0938b-83ae-44a8-82ac-21d1b8d1a505 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.287854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0938b-83ae-44a8-82ac-21d1b8d1a505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.288200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0938b-83ae-44a8-82ac-21d1b8d1a505 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.288969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0938b-83ae-44a8-82ac-21d1b8d1a505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:20.291675 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0938b-83ae-44a8-82ac-21d1b8d1a505 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:20.300205 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae662a4-fb77-444e-9b1b-bbbabe94aa0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bcdbcfc080f4b09afb2ba8985e0bf86, user_id=92017633cf094918bbdad11d0a1f79a5, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:20.304124 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae662a4-fb77-444e-9b1b-bbbabe94aa0e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:20.319647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 921/1844] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:20 2022] PUT /identity/v3/projects/3bcdbcfc080f4b09afb2ba8985e0bf86/users/92017633cf094918bbdad11d0a1f79a5/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:20.324616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 924/1845] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:20 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:13:20.325584 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae662a4-fb77-444e-9b1b-bbbabe94aa0e admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:20.331439 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90534b3-76e2-4194-a0cd-e28db96f13cf None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.331818 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90534b3-76e2-4194-a0cd-e28db96f13cf None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.332174 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90534b3-76e2-4194-a0cd-e28db96f13cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.364978 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-f90534b3-76e2-4194-a0cd-e28db96f13cf None None] MFA Rules not processed for user `92017633cf094918bbdad11d0a1f79a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:20.366266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f90534b3-76e2-4194-a0cd-e28db96f13cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:20.405874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae662a4-fb77-444e-9b1b-bbbabe94aa0e admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:20.426382 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:20.447812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 925/1846] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:20.449076 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90534b3-76e2-4194-a0cd-e28db96f13cf None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:20.450369 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f90534b3-76e2-4194-a0cd-e28db96f13cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:20.467985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b620d-bf0e-465b-b1ab-445473a1a0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:20.469443 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b620d-bf0e-465b-b1ab-445473a1a0d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.469879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b620d-bf0e-465b-b1ab-445473a1a0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.470218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b620d-bf0e-465b-b1ab-445473a1a0d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.470949 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b620d-bf0e-465b-b1ab-445473a1a0d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:20.472848 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28b620d-bf0e-465b-b1ab-445473a1a0d3 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:20.492685 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 922/1847] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:20 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:13:20.492685 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b620d-bf0e-465b-b1ab-445473a1a0d3 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:20.503120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28b620d-bf0e-465b-b1ab-445473a1a0d3 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:20.507812 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0099718-9561-4559-bd61-3548f1e16905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:20.508454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0099718-9561-4559-bd61-3548f1e16905 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.508908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0099718-9561-4559-bd61-3548f1e16905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.509187 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0099718-9561-4559-bd61-3548f1e16905 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.509992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0099718-9561-4559-bd61-3548f1e16905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:20.511886 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0099718-9561-4559-bd61-3548f1e16905 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:20.519598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90534b3-76e2-4194-a0cd-e28db96f13cf None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:20.524808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:20.525777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.526091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.526375 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587 admin admin] PATH_INFO: `/v3/projects/f4549a677ce640ca96d33aeedd9abb40` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.527598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4549a677ce640ca96d33aeedd9abb40)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:20.529528 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:20.538291 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 923/1848] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:20 2022] POST /identity/v3/projects => generated 393 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:20.539153 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0099718-9561-4559-bd61-3548f1e16905 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:20.584563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4549a677ce640ca96d33aeedd9abb40', 'request_id': 'req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:20.589805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:20.591506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4549a677ce640ca96d33aeedd9abb40', 'request_id': 'req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:20.600814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4549a677ce640ca96d33aeedd9abb40', 'request_id': 'req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:20.604649 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0099718-9561-4559-bd61-3548f1e16905 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:20.613078 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587 admin admin] The token cache is being invalidate because project f4549a677ce640ca96d33aeedd9abb40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:20.613368 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:20.614022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:20.615841 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 926/1849] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:20 2022] DELETE /identity/v3/projects/f4549a677ce640ca96d33aeedd9abb40 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:20.616542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:20.617864 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:20.661592 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29378a47-4e0d-46e9-880c-f70b3b4a904d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:20.662271 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378a47-4e0d-46e9-880c-f70b3b4a904d admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.662554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378a47-4e0d-46e9-880c-f70b3b4a904d admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.662865 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378a47-4e0d-46e9-880c-f70b3b4a904d admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.663596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29378a47-4e0d-46e9-880c-f70b3b4a904d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:20.666061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29378a47-4e0d-46e9-880c-f70b3b4a904d admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:20.675951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 924/1850] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:20.677207 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ea89b-5cb9-43b2-bee8-c2ea01ce20a8 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.677836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ea89b-5cb9-43b2-bee8-c2ea01ce20a8 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.678130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ea89b-5cb9-43b2-bee8-c2ea01ce20a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.682711 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4ad27-39bc-4f0c-bbfe-80a675c9b587 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:20.687457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc34488-adb1-4be9-820d-0c2ac8d3feaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:20.688326 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc34488-adb1-4be9-820d-0c2ac8d3feaa admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.688618 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc34488-adb1-4be9-820d-0c2ac8d3feaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.688898 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc34488-adb1-4be9-820d-0c2ac8d3feaa admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.689583 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc34488-adb1-4be9-820d-0c2ac8d3feaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:20.691554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc34488-adb1-4be9-820d-0c2ac8d3feaa admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:20.697711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-c34ea89b-5cb9-43b2-bee8-c2ea01ce20a8 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:20.699366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c34ea89b-5cb9-43b2-bee8-c2ea01ce20a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:20.715848 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 925/1851] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:20.715848 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ea89b-5cb9-43b2-bee8-c2ea01ce20a8 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:20.715848 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c34ea89b-5cb9-43b2-bee8-c2ea01ce20a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:20.716192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 927/1852] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:20 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:20.716997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc34488-adb1-4be9-820d-0c2ac8d3feaa admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:20.724530 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ea89b-5cb9-43b2-bee8-c2ea01ce20a8 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:20.727950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc34488-adb1-4be9-820d-0c2ac8d3feaa admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:20.729671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58738bf-8161-4b37-8aa1-a27436ba531c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:20.730562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58738bf-8161-4b37-8aa1-a27436ba531c admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.730975 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58738bf-8161-4b37-8aa1-a27436ba531c admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.731259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58738bf-8161-4b37-8aa1-a27436ba531c admin admin] PATH_INFO: `/v3/projects/4446e63c2cd145468c4ed36802e5e486/users/4c19880afeae46a1a56e0cf683f44d12/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.733663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4244ded-d0f6-4e43-ae1e-f94450862de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:20.733865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4244ded-d0f6-4e43-ae1e-f94450862de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.734266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4244ded-d0f6-4e43-ae1e-f94450862de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.734551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4244ded-d0f6-4e43-ae1e-f94450862de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.735368 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4244ded-d0f6-4e43-ae1e-f94450862de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:20.736981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4244ded-d0f6-4e43-ae1e-f94450862de2 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:20.747484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58738bf-8161-4b37-8aa1-a27436ba531c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4446e63c2cd145468c4ed36802e5e486, user_id=4c19880afeae46a1a56e0cf683f44d12, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:20.750643 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58738bf-8161-4b37-8aa1-a27436ba531c admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:20.754738 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 928/1853] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:20.756396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21922ffb-4d6e-4865-a6e6-9e7d173f4e7e None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.756966 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21922ffb-4d6e-4865-a6e6-9e7d173f4e7e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.757378 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21922ffb-4d6e-4865-a6e6-9e7d173f4e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.781036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 926/1854] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:20 2022] PUT /identity/v3/projects/4446e63c2cd145468c4ed36802e5e486/users/4c19880afeae46a1a56e0cf683f44d12/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:20.782066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58738bf-8161-4b37-8aa1-a27436ba531c admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:20.783649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-21922ffb-4d6e-4865-a6e6-9e7d173f4e7e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:20.784811 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-21922ffb-4d6e-4865-a6e6-9e7d173f4e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:20.793040 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:20.865434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58738bf-8161-4b37-8aa1-a27436ba531c admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:20.870589 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55bf6b-e511-4950-80e0-db4b099bd2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:20.871524 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55bf6b-e511-4950-80e0-db4b099bd2fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.871799 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55bf6b-e511-4950-80e0-db4b099bd2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.872314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55bf6b-e511-4950-80e0-db4b099bd2fd admin admin] PATH_INFO: `/v3/projects/50c91168984b4c0f8a127f8ad345befb/users/1b3f114233064392af6a7e81a676d0b5/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.872426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 929/1855] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:20.873123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21922ffb-4d6e-4865-a6e6-9e7d173f4e7e None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:20.883275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21922ffb-4d6e-4865-a6e6-9e7d173f4e7e None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:20.890463 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee55bf6b-e511-4950-80e0-db4b099bd2fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c91168984b4c0f8a127f8ad345befb, user_id=1b3f114233064392af6a7e81a676d0b5, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:20.890644 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90491d-a097-401d-b727-5fa60306b4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:20.891390 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90491d-a097-401d-b727-5fa60306b4cf admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:20.891746 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90491d-a097-401d-b727-5fa60306b4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:20.892090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90491d-a097-401d-b727-5fa60306b4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:20.892961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90491d-a097-401d-b727-5fa60306b4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:20.893612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee55bf6b-e511-4950-80e0-db4b099bd2fd admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:20.894776 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90491d-a097-401d-b727-5fa60306b4cf admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:20.908543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 930/1856] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:20.912000 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 927/1857] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:20 2022] PUT /identity/v3/projects/50c91168984b4c0f8a127f8ad345befb/users/1b3f114233064392af6a7e81a676d0b5/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:20.912674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90491d-a097-401d-b727-5fa60306b4cf admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:20.916353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55bf6b-e511-4950-80e0-db4b099bd2fd admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:20.925079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90491d-a097-401d-b727-5fa60306b4cf admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:20.994851 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:21.004023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55bf6b-e511-4950-80e0-db4b099bd2fd admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:21.007042 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed8873-077d-41f5-af0b-edbdad774496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.008088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed8873-077d-41f5-af0b-edbdad774496 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.009255 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed8873-077d-41f5-af0b-edbdad774496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.009583 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed8873-077d-41f5-af0b-edbdad774496 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.010369 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed8873-077d-41f5-af0b-edbdad774496 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:21.010483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed54374-d9be-4ebd-b63c-484dcb6d9dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.011158 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed8873-077d-41f5-af0b-edbdad774496 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:21.011359 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed54374-d9be-4ebd-b63c-484dcb6d9dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.011642 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed54374-d9be-4ebd-b63c-484dcb6d9dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.011919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed54374-d9be-4ebd-b63c-484dcb6d9dc0 admin admin] PATH_INFO: `/v3/projects/4446e63c2cd145468c4ed36802e5e486/users/4c19880afeae46a1a56e0cf683f44d12/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.012783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ed8873-077d-41f5-af0b-edbdad774496 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:21.013998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed54374-d9be-4ebd-b63c-484dcb6d9dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4446e63c2cd145468c4ed36802e5e486, user_id=4c19880afeae46a1a56e0cf683f44d12, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:21.016659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed54374-d9be-4ebd-b63c-484dcb6d9dc0 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:21.031213 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 931/1859] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:13:21.032202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed8873-077d-41f5-af0b-edbdad774496 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:21.038733 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 928/1859] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:20 2022] PUT /identity/v3/projects/4446e63c2cd145468c4ed36802e5e486/users/4c19880afeae46a1a56e0cf683f44d12/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:21.040305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed54374-d9be-4ebd-b63c-484dcb6d9dc0 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:21.048972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed8873-077d-41f5-af0b-edbdad774496 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:21.119885 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175a317-7070-434d-aa08-52022d7615b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.120602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175a317-7070-434d-aa08-52022d7615b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.121055 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175a317-7070-434d-aa08-52022d7615b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.121328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175a317-7070-434d-aa08-52022d7615b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.122643 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175a317-7070-434d-aa08-52022d7615b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:21.124617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175a317-7070-434d-aa08-52022d7615b7 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:21.131511 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed54374-d9be-4ebd-b63c-484dcb6d9dc0 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:21.136994 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e7ef2-c18c-4518-a1e8-addaf3cebbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.137730 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e7ef2-c18c-4518-a1e8-addaf3cebbe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.138019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e7ef2-c18c-4518-a1e8-addaf3cebbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.138297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e7ef2-c18c-4518-a1e8-addaf3cebbe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.139023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e7ef2-c18c-4518-a1e8-addaf3cebbe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:21.141594 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 932/1860] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:21.141756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e7ef2-c18c-4518-a1e8-addaf3cebbe4 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:21.142327 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966e8fd-52c5-4020-88be-f2c52720762b None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.142781 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966e8fd-52c5-4020-88be-f2c52720762b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.143068 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966e8fd-52c5-4020-88be-f2c52720762b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.173933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 929/1861] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:21 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:21.173933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e7ef2-c18c-4518-a1e8-addaf3cebbe4 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:21.174203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-8966e8fd-52c5-4020-88be-f2c52720762b None None] MFA Rules not processed for user `4c19880afeae46a1a56e0cf683f44d12`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:21.175500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8966e8fd-52c5-4020-88be-f2c52720762b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:21.182157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:21.260201 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e7ef2-c18c-4518-a1e8-addaf3cebbe4 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:21.264896 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfd434-5e92-4ae9-855d-935723dac0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.265868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfd434-5e92-4ae9-855d-935723dac0ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.266152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfd434-5e92-4ae9-855d-935723dac0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.266433 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfd434-5e92-4ae9-855d-935723dac0ed admin admin] PATH_INFO: `/v3/projects/50c91168984b4c0f8a127f8ad345befb/users/1b3f114233064392af6a7e81a676d0b5/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.268383 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 933/1862] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:21 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:21.268498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfd434-5e92-4ae9-855d-935723dac0ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c91168984b4c0f8a127f8ad345befb, user_id=1b3f114233064392af6a7e81a676d0b5, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:21.269319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966e8fd-52c5-4020-88be-f2c52720762b None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:21.271759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfd434-5e92-4ae9-855d-935723dac0ed admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:21.280942 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966e8fd-52c5-4020-88be-f2c52720762b None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:21.283343 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 930/1863] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:21 2022] PUT /identity/v3/projects/50c91168984b4c0f8a127f8ad345befb/users/1b3f114233064392af6a7e81a676d0b5/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:21.284225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfd434-5e92-4ae9-855d-935723dac0ed admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:21.286867 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b3cfd434-5e92-4ae9-855d-935723dac0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:21.339143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae26bf1-dedb-4304-9e0e-81e5fa4c959f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.339825 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae26bf1-dedb-4304-9e0e-81e5fa4c959f admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.340110 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae26bf1-dedb-4304-9e0e-81e5fa4c959f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.340391 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae26bf1-dedb-4304-9e0e-81e5fa4c959f admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.341091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae26bf1-dedb-4304-9e0e-81e5fa4c959f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:21.343045 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae26bf1-dedb-4304-9e0e-81e5fa4c959f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:21.344303 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfd434-5e92-4ae9-855d-935723dac0ed admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:21.348664 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be368ec2-0fdd-4145-b0b8-0deaca70b778 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.349410 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be368ec2-0fdd-4145-b0b8-0deaca70b778 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.349813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be368ec2-0fdd-4145-b0b8-0deaca70b778 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.350101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be368ec2-0fdd-4145-b0b8-0deaca70b778 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.359555 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:21.370912 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 934/1864] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:21 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:21.372072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405a34d-5b7e-4567-a81e-bfd401886d25 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.372366 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405a34d-5b7e-4567-a81e-bfd401886d25 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.372644 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405a34d-5b7e-4567-a81e-bfd401886d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.399227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-9405a34d-5b7e-4567-a81e-bfd401886d25 None None] MFA Rules not processed for user `1b3f114233064392af6a7e81a676d0b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:21.400936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-9405a34d-5b7e-4567-a81e-bfd401886d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:21.415101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be368ec2-0fdd-4145-b0b8-0deaca70b778 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:21.416572 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be368ec2-0fdd-4145-b0b8-0deaca70b778 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:21.426348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 931/1865] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:21.426969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be368ec2-0fdd-4145-b0b8-0deaca70b778 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:21.437456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be368ec2-0fdd-4145-b0b8-0deaca70b778 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:21.442459 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00e6f1-b848-4060-ae84-4d360f130fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.443514 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00e6f1-b848-4060-ae84-4d360f130fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.444108 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00e6f1-b848-4060-ae84-4d360f130fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.444108 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00e6f1-b848-4060-ae84-4d360f130fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.444791 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00e6f1-b848-4060-ae84-4d360f130fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:21.447159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00e6f1-b848-4060-ae84-4d360f130fe9 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:21.462723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 932/1866] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:21.464085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00e6f1-b848-4060-ae84-4d360f130fe9 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:21.475454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00e6f1-b848-4060-ae84-4d360f130fe9 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:21.479522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 935/1867] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:21.481981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743930a-67d4-4eb0-97e2-d84bbd023f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.483213 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743930a-67d4-4eb0-97e2-d84bbd023f7b admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.483663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743930a-67d4-4eb0-97e2-d84bbd023f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.484085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743930a-67d4-4eb0-97e2-d84bbd023f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.485172 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3743930a-67d4-4eb0-97e2-d84bbd023f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:21.487165 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3743930a-67d4-4eb0-97e2-d84bbd023f7b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:21.490190 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405a34d-5b7e-4567-a81e-bfd401886d25 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:21.500241 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405a34d-5b7e-4567-a81e-bfd401886d25 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:21.503954 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a809dd-e24b-4624-a528-7835e6ba412a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.504599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a809dd-e24b-4624-a528-7835e6ba412a service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.504997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a809dd-e24b-4624-a528-7835e6ba412a service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.505129 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a809dd-e24b-4624-a528-7835e6ba412a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.506857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 933/1868] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:21.508101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743930a-67d4-4eb0-97e2-d84bbd023f7b admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:21.517557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a809dd-e24b-4624-a528-7835e6ba412a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:21.518962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743930a-67d4-4eb0-97e2-d84bbd023f7b admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:21.519069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a809dd-e24b-4624-a528-7835e6ba412a service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:21.523555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98b9cd-68a9-4d34-b2ed-edab5bdc8146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.525015 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98b9cd-68a9-4d34-b2ed-edab5bdc8146 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.525297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98b9cd-68a9-4d34-b2ed-edab5bdc8146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.525573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98b9cd-68a9-4d34-b2ed-edab5bdc8146 admin admin] PATH_INFO: `/v3/projects/88e112091fb645e5882097d1459248cd/users/802d1e9115d6487f90468bab7f21027d/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.529521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 936/1869] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:21.532002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a809dd-e24b-4624-a528-7835e6ba412a service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:21.533271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b2a809dd-e24b-4624-a528-7835e6ba412a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:21.540632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98b9cd-68a9-4d34-b2ed-edab5bdc8146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e112091fb645e5882097d1459248cd, user_id=802d1e9115d6487f90468bab7f21027d, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:21.543302 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98b9cd-68a9-4d34-b2ed-edab5bdc8146 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:21.545766 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a809dd-e24b-4624-a528-7835e6ba412a service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:21.551234 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f98d6-acbf-451e-85ad-e7a7381069f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.551859 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f98d6-acbf-451e-85ad-e7a7381069f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.552147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f98d6-acbf-451e-85ad-e7a7381069f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.552426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f98d6-acbf-451e-85ad-e7a7381069f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.555550 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-037f98d6-acbf-451e-85ad-e7a7381069f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:21.561490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 934/1870] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:21 2022] PUT /identity/v3/projects/88e112091fb645e5882097d1459248cd/users/802d1e9115d6487f90468bab7f21027d/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:21.576285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787976d-74a2-414e-82b3-502b094b1012 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.576680 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787976d-74a2-414e-82b3-502b094b1012 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.576852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787976d-74a2-414e-82b3-502b094b1012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.582741 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:21.611342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-6787976d-74a2-414e-82b3-502b094b1012 None None] MFA Rules not processed for user `802d1e9115d6487f90468bab7f21027d`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:21.612656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6787976d-74a2-414e-82b3-502b094b1012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:21.624979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f98d6-acbf-451e-85ad-e7a7381069f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:21.626342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f98d6-acbf-451e-85ad-e7a7381069f5 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:21.636194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 937/1871] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:21 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:21.637051 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f98d6-acbf-451e-85ad-e7a7381069f5 service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:21.644915 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f98d6-acbf-451e-85ad-e7a7381069f5 service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:21.694100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 935/1872] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:21.695440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787976d-74a2-414e-82b3-502b094b1012 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:21.707426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aac397-1394-4936-a7d1-405e2d3e3e43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.708318 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aac397-1394-4936-a7d1-405e2d3e3e43 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.708672 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aac397-1394-4936-a7d1-405e2d3e3e43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.709022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aac397-1394-4936-a7d1-405e2d3e3e43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.711110 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c4aac397-1394-4936-a7d1-405e2d3e3e43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:21.736119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:21.768748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787976d-74a2-414e-82b3-502b094b1012 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:21.773995 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fc53e-7d32-4dc7-97eb-4ff81750f596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.775658 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fc53e-7d32-4dc7-97eb-4ff81750f596 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.776022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fc53e-7d32-4dc7-97eb-4ff81750f596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.776378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fc53e-7d32-4dc7-97eb-4ff81750f596 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.777673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fc53e-7d32-4dc7-97eb-4ff81750f596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:21.780460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fc53e-7d32-4dc7-97eb-4ff81750f596 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:21.781548 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aac397-1394-4936-a7d1-405e2d3e3e43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:21.783103 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aac397-1394-4936-a7d1-405e2d3e3e43 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:21.794597 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 938/1873] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:21.813499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 936/1874] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:21 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:13:21.819636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aac397-1394-4936-a7d1-405e2d3e3e43 service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:21.831376 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aac397-1394-4936-a7d1-405e2d3e3e43 service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:21.890333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d905f-4ff2-410d-89bd-dde3461a31cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.891064 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d905f-4ff2-410d-89bd-dde3461a31cf admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.891346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d905f-4ff2-410d-89bd-dde3461a31cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.891624 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d905f-4ff2-410d-89bd-dde3461a31cf admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.892308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d905f-4ff2-410d-89bd-dde3461a31cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:21.894270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d905f-4ff2-410d-89bd-dde3461a31cf admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:21.915794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 939/1875] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:21 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:21.919812 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fc53e-7d32-4dc7-97eb-4ff81750f596 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:21.931041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fc53e-7d32-4dc7-97eb-4ff81750f596 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:21.936400 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1628e5-6752-465f-b79b-fdf210bb313b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.937171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1628e5-6752-465f-b79b-fdf210bb313b admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.937493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1628e5-6752-465f-b79b-fdf210bb313b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.937840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1628e5-6752-465f-b79b-fdf210bb313b admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:21.938603 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1628e5-6752-465f-b79b-fdf210bb313b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:21.940394 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1628e5-6752-465f-b79b-fdf210bb313b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:21.954342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 937/1876] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:21.959232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d905f-4ff2-410d-89bd-dde3461a31cf admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:21.974887 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d905f-4ff2-410d-89bd-dde3461a31cf admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:21.980761 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d8c6e-0656-45da-a431-e6624d0f8819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:21.981740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d8c6e-0656-45da-a431-e6624d0f8819 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:21.982047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d8c6e-0656-45da-a431-e6624d0f8819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:21.982321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d8c6e-0656-45da-a431-e6624d0f8819 admin admin] PATH_INFO: `/v3/projects/fe6bb08da48948d88daa43ee433bba64/users/368f0ff2a9eb41008cfd97951f33ded3/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:22.027830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d8c6e-0656-45da-a431-e6624d0f8819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe6bb08da48948d88daa43ee433bba64, user_id=368f0ff2a9eb41008cfd97951f33ded3, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:22.031154 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d8c6e-0656-45da-a431-e6624d0f8819 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:22.050298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 940/1877] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:21 2022] PUT /identity/v3/projects/fe6bb08da48948d88daa43ee433bba64/users/368f0ff2a9eb41008cfd97951f33ded3/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:22.054062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1628e5-6752-465f-b79b-fdf210bb313b admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:22.065613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1628e5-6752-465f-b79b-fdf210bb313b admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:22.178381 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0088ca7-835a-4484-8426-a1bbaa205b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:22.179266 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0088ca7-835a-4484-8426-a1bbaa205b92 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:22.179644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0088ca7-835a-4484-8426-a1bbaa205b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:22.179926 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0088ca7-835a-4484-8426-a1bbaa205b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:22.181391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0088ca7-835a-4484-8426-a1bbaa205b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:22.183464 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0088ca7-835a-4484-8426-a1bbaa205b92 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:22.194720 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 938/1878] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:22.198575 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d8c6e-0656-45da-a431-e6624d0f8819 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:22.209680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d8c6e-0656-45da-a431-e6624d0f8819 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:22.222183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eeb17c-cc2b-4044-a06e-25dfb2d68909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:22.222183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eeb17c-cc2b-4044-a06e-25dfb2d68909 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:22.222183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eeb17c-cc2b-4044-a06e-25dfb2d68909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:22.222183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eeb17c-cc2b-4044-a06e-25dfb2d68909 admin admin] PATH_INFO: `/v3/projects/fe6bb08da48948d88daa43ee433bba64/users/368f0ff2a9eb41008cfd97951f33ded3/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:22.222541 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eeb17c-cc2b-4044-a06e-25dfb2d68909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe6bb08da48948d88daa43ee433bba64, user_id=368f0ff2a9eb41008cfd97951f33ded3, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:22.222541 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eeb17c-cc2b-4044-a06e-25dfb2d68909 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:22.234737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 941/1879] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:22 2022] PUT /identity/v3/projects/fe6bb08da48948d88daa43ee433bba64/users/368f0ff2a9eb41008cfd97951f33ded3/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:22.245075 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c81766-e08c-4cf4-9e0d-76f7caa0d9af None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:22.245372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c81766-e08c-4cf4-9e0d-76f7caa0d9af None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:22.245667 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c81766-e08c-4cf4-9e0d-76f7caa0d9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:22.251479 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:22.276910 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-36c81766-e08c-4cf4-9e0d-76f7caa0d9af None None] MFA Rules not processed for user `368f0ff2a9eb41008cfd97951f33ded3`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:22.278741 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-36c81766-e08c-4cf4-9e0d-76f7caa0d9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:22.378089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 939/1880] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:22.411549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eeb17c-cc2b-4044-a06e-25dfb2d68909 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:22.517413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eeb17c-cc2b-4044-a06e-25dfb2d68909 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:22.521647 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc408b-4104-4d5d-b815-f91a429c37d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:22.523425 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc408b-4104-4d5d-b815-f91a429c37d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:22.523822 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc408b-4104-4d5d-b815-f91a429c37d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:22.525278 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc408b-4104-4d5d-b815-f91a429c37d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:22.540436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc408b-4104-4d5d-b815-f91a429c37d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:22.543245 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc408b-4104-4d5d-b815-f91a429c37d7 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:22.557264 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 942/1881] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:22.634150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c81766-e08c-4cf4-9e0d-76f7caa0d9af None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:22.711567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c81766-e08c-4cf4-9e0d-76f7caa0d9af None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:22.717116 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c0d47-b6fe-4430-8262-1ca87a32705b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:22.718007 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c0d47-b6fe-4430-8262-1ca87a32705b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:22.718426 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c0d47-b6fe-4430-8262-1ca87a32705b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:22.718735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c0d47-b6fe-4430-8262-1ca87a32705b admin admin] PATH_INFO: `/v3/users/802d1e9115d6487f90468bab7f21027d` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:22.719950 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c0d47-b6fe-4430-8262-1ca87a32705b admin admin] RBAC: Authorizing `identity:delete_user(user_id=802d1e9115d6487f90468bab7f21027d)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:22.722149 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c0d47-b6fe-4430-8262-1ca87a32705b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:22.774618 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-798c0d47-b6fe-4430-8262-1ca87a32705b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '802d1e9115d6487f90468bab7f21027d'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:22.787227 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-798c0d47-b6fe-4430-8262-1ca87a32705b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '802d1e9115d6487f90468bab7f21027d'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:22.796398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-798c0d47-b6fe-4430-8262-1ca87a32705b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '802d1e9115d6487f90468bab7f21027d'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:22.797154 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-798c0d47-b6fe-4430-8262-1ca87a32705b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '802d1e9115d6487f90468bab7f21027d'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:22.807169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-798c0d47-b6fe-4430-8262-1ca87a32705b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '802d1e9115d6487f90468bab7f21027d'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:22.813990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 940/1882] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:22 2022] DELETE /identity/v3/users/802d1e9115d6487f90468bab7f21027d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:22.818877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d77ccb-530b-4b45-8e70-ce9aa2354531 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:22.819274 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d77ccb-530b-4b45-8e70-ce9aa2354531 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:22.819706 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d77ccb-530b-4b45-8e70-ce9aa2354531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:22.856662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-84d77ccb-530b-4b45-8e70-ce9aa2354531 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:22.858074 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-84d77ccb-530b-4b45-8e70-ce9aa2354531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:22.874920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:22.952515 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 943/1883] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:22.965337 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c0d47-b6fe-4430-8262-1ca87a32705b admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:22.966984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-798c0d47-b6fe-4430-8262-1ca87a32705b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:22.978335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:23.045333 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c0d47-b6fe-4430-8262-1ca87a32705b admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:23.049949 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1284dfd-7e56-4870-9c98-6862bcff2758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:23.050866 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1284dfd-7e56-4870-9c98-6862bcff2758 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:23.051444 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1284dfd-7e56-4870-9c98-6862bcff2758 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:23.051939 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1284dfd-7e56-4870-9c98-6862bcff2758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:23.068317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1284dfd-7e56-4870-9c98-6862bcff2758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:23.070993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1284dfd-7e56-4870-9c98-6862bcff2758 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:23.087745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 941/1884] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:23.837305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d77ccb-530b-4b45-8e70-ce9aa2354531 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:23.839329 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-84d77ccb-530b-4b45-8e70-ce9aa2354531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:23.865953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d77ccb-530b-4b45-8e70-ce9aa2354531 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:23.878081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d2143-2473-439b-bcfb-49c8aacc317c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:23.879052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d2143-2473-439b-bcfb-49c8aacc317c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:23.879268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d2143-2473-439b-bcfb-49c8aacc317c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:23.879546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d2143-2473-439b-bcfb-49c8aacc317c admin admin] PATH_INFO: `/v3/projects/88e112091fb645e5882097d1459248cd` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:23.882871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d2143-2473-439b-bcfb-49c8aacc317c admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e112091fb645e5882097d1459248cd)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:23.886513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d2143-2473-439b-bcfb-49c8aacc317c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:24.046727 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-307d2143-2473-439b-bcfb-49c8aacc317c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e112091fb645e5882097d1459248cd', 'request_id': 'req-307d2143-2473-439b-bcfb-49c8aacc317c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:24.054578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-307d2143-2473-439b-bcfb-49c8aacc317c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e112091fb645e5882097d1459248cd', 'request_id': 'req-307d2143-2473-439b-bcfb-49c8aacc317c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:24.076057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-307d2143-2473-439b-bcfb-49c8aacc317c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e112091fb645e5882097d1459248cd', 'request_id': 'req-307d2143-2473-439b-bcfb-49c8aacc317c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:24.104462 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-307d2143-2473-439b-bcfb-49c8aacc317c admin admin] The token cache is being invalidate because project 88e112091fb645e5882097d1459248cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:24.105232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-307d2143-2473-439b-bcfb-49c8aacc317c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:24.106124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-307d2143-2473-439b-bcfb-49c8aacc317c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:24.108495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 944/1885] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:23 2022] DELETE /identity/v3/projects/88e112091fb645e5882097d1459248cd => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:24.112607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1284dfd-7e56-4870-9c98-6862bcff2758 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:24.125565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1284dfd-7e56-4870-9c98-6862bcff2758 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:24.347051 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31922b60-17c8-455b-91a6-a772e5c6ff2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:24.349027 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31922b60-17c8-455b-91a6-a772e5c6ff2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:24.349317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31922b60-17c8-455b-91a6-a772e5c6ff2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:24.349594 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31922b60-17c8-455b-91a6-a772e5c6ff2d admin admin] PATH_INFO: `/v3/users/368f0ff2a9eb41008cfd97951f33ded3` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:24.361466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31922b60-17c8-455b-91a6-a772e5c6ff2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=368f0ff2a9eb41008cfd97951f33ded3)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:24.363949 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31922b60-17c8-455b-91a6-a772e5c6ff2d admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:24.437734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-31922b60-17c8-455b-91a6-a772e5c6ff2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '368f0ff2a9eb41008cfd97951f33ded3'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:24.456482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-31922b60-17c8-455b-91a6-a772e5c6ff2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '368f0ff2a9eb41008cfd97951f33ded3'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:24.469198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-31922b60-17c8-455b-91a6-a772e5c6ff2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '368f0ff2a9eb41008cfd97951f33ded3'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:24.471421 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-31922b60-17c8-455b-91a6-a772e5c6ff2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '368f0ff2a9eb41008cfd97951f33ded3'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:24.488040 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-31922b60-17c8-455b-91a6-a772e5c6ff2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '368f0ff2a9eb41008cfd97951f33ded3'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:24.505474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 942/1886] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:24 2022] DELETE /identity/v3/users/368f0ff2a9eb41008cfd97951f33ded3 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:24.886857 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d2143-2473-439b-bcfb-49c8aacc317c admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:24.889133 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-307d2143-2473-439b-bcfb-49c8aacc317c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:24.905722 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d2143-2473-439b-bcfb-49c8aacc317c admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:24.984596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68181806-4cca-4c60-ad3d-ac1d376643a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:24.987522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68181806-4cca-4c60-ad3d-ac1d376643a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:24.987889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68181806-4cca-4c60-ad3d-ac1d376643a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:24.988244 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68181806-4cca-4c60-ad3d-ac1d376643a5 admin admin] PATH_INFO: `/v3/users/3e970d8238614224b0b7f82801ce5382` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:24.990446 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68181806-4cca-4c60-ad3d-ac1d376643a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e970d8238614224b0b7f82801ce5382)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:24.993233 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68181806-4cca-4c60-ad3d-ac1d376643a5 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:25.047511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:25.050707 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-68181806-4cca-4c60-ad3d-ac1d376643a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e970d8238614224b0b7f82801ce5382'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:25.067701 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-68181806-4cca-4c60-ad3d-ac1d376643a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e970d8238614224b0b7f82801ce5382'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:25.079767 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-68181806-4cca-4c60-ad3d-ac1d376643a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e970d8238614224b0b7f82801ce5382'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:25.080771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-68181806-4cca-4c60-ad3d-ac1d376643a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e970d8238614224b0b7f82801ce5382'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:25.103978 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-68181806-4cca-4c60-ad3d-ac1d376643a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e970d8238614224b0b7f82801ce5382'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:25.111261 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 945/1887] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:24 2022] DELETE /identity/v3/users/3e970d8238614224b0b7f82801ce5382 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:25.115857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956483c-74a4-4b50-9c0e-5cce04c2ac44 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:25.116342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956483c-74a4-4b50-9c0e-5cce04c2ac44 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:25.117088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956483c-74a4-4b50-9c0e-5cce04c2ac44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:25.134223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-f956483c-74a4-4b50-9c0e-5cce04c2ac44 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:25.135609 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f956483c-74a4-4b50-9c0e-5cce04c2ac44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:25.160934 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:25.212236 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 943/1888] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:25.224058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68181806-4cca-4c60-ad3d-ac1d376643a5 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:25.226369 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-68181806-4cca-4c60-ad3d-ac1d376643a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:25.228872 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956483c-74a4-4b50-9c0e-5cce04c2ac44 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:25.232468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f956483c-74a4-4b50-9c0e-5cce04c2ac44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:25.249797 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68181806-4cca-4c60-ad3d-ac1d376643a5 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:25.267668 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af578d6f-750a-40f7-be56-45f5180dbe8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:25.268566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af578d6f-750a-40f7-be56-45f5180dbe8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:25.269416 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af578d6f-750a-40f7-be56-45f5180dbe8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:25.269706 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af578d6f-750a-40f7-be56-45f5180dbe8d admin admin] PATH_INFO: `/v3/projects/fe6bb08da48948d88daa43ee433bba64` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:25.285488 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af578d6f-750a-40f7-be56-45f5180dbe8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe6bb08da48948d88daa43ee433bba64)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:25.288227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af578d6f-750a-40f7-be56-45f5180dbe8d admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:25.342908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956483c-74a4-4b50-9c0e-5cce04c2ac44 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:25.353464 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ddc9d-d761-4261-b5fa-14204d068bff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:25.354352 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ddc9d-d761-4261-b5fa-14204d068bff service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:25.354778 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ddc9d-d761-4261-b5fa-14204d068bff service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:25.355638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ddc9d-d761-4261-b5fa-14204d068bff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:25.370288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ddc9d-d761-4261-b5fa-14204d068bff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:25.372040 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ddc9d-d761-4261-b5fa-14204d068bff service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:25.386171 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-af578d6f-750a-40f7-be56-45f5180dbe8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe6bb08da48948d88daa43ee433bba64', 'request_id': 'req-af578d6f-750a-40f7-be56-45f5180dbe8d'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:25.388534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 944/1889] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:25.394636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-af578d6f-750a-40f7-be56-45f5180dbe8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe6bb08da48948d88daa43ee433bba64', 'request_id': 'req-af578d6f-750a-40f7-be56-45f5180dbe8d'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:25.406208 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-af578d6f-750a-40f7-be56-45f5180dbe8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe6bb08da48948d88daa43ee433bba64', 'request_id': 'req-af578d6f-750a-40f7-be56-45f5180dbe8d'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:25.430326 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-af578d6f-750a-40f7-be56-45f5180dbe8d admin admin] The token cache is being invalidate because project fe6bb08da48948d88daa43ee433bba64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:25.431780 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-af578d6f-750a-40f7-be56-45f5180dbe8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:25.432891 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-af578d6f-750a-40f7-be56-45f5180dbe8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:25.438809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 946/1890] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:25 2022] DELETE /identity/v3/projects/fe6bb08da48948d88daa43ee433bba64 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:25.440336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae861a82-a542-4c9c-9fc2-3c5802bfcce4 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:25.440632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae861a82-a542-4c9c-9fc2-3c5802bfcce4 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:25.440910 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae861a82-a542-4c9c-9fc2-3c5802bfcce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:25.462229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-ae861a82-a542-4c9c-9fc2-3c5802bfcce4 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:25.463972 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ae861a82-a542-4c9c-9fc2-3c5802bfcce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:25.594507 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 945/1891] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:25.604151 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62286beb-3af9-49a1-807f-849f4da0c336 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:25.604487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62286beb-3af9-49a1-807f-849f4da0c336 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:25.604779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62286beb-3af9-49a1-807f-849f4da0c336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:25.626232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-62286beb-3af9-49a1-807f-849f4da0c336 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:25.627612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-62286beb-3af9-49a1-807f-849f4da0c336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:25.639475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 947/1892] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:25.646158 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae861a82-a542-4c9c-9fc2-3c5802bfcce4 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:25.655885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae861a82-a542-4c9c-9fc2-3c5802bfcce4 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:25.662410 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246f9a9-db9e-4a94-a99e-355468521893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:25.663207 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246f9a9-db9e-4a94-a99e-355468521893 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:25.663569 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246f9a9-db9e-4a94-a99e-355468521893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:25.663922 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0246f9a9-db9e-4a94-a99e-355468521893 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:25.665357 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246f9a9-db9e-4a94-a99e-355468521893 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:25.666129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246f9a9-db9e-4a94-a99e-355468521893 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:25.667670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0246f9a9-db9e-4a94-a99e-355468521893 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:25.683780 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 946/1893] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:13:25.687888 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62286beb-3af9-49a1-807f-849f4da0c336 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:25.712436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62286beb-3af9-49a1-807f-849f4da0c336 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:25.722982 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc4e4c-e8cd-419b-b850-73d94047ecfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:25.723863 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc4e4c-e8cd-419b-b850-73d94047ecfd admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:25.724557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc4e4c-e8cd-419b-b850-73d94047ecfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:25.725690 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc4e4c-e8cd-419b-b850-73d94047ecfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:25.726479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc4e4c-e8cd-419b-b850-73d94047ecfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:25.731285 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc4e4c-e8cd-419b-b850-73d94047ecfd admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:25.793727 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 948/1894] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:25 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:25.797688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246f9a9-db9e-4a94-a99e-355468521893 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:25.816171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0246f9a9-db9e-4a94-a99e-355468521893 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:25.869747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc4e4c-e8cd-419b-b850-73d94047ecfd admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:25.872058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b8cc4e4c-e8cd-419b-b850-73d94047ecfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:25.953158 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8c9a3-0b17-4325-8fb5-42df5a883628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:25.954016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8c9a3-0b17-4325-8fb5-42df5a883628 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:25.954279 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8c9a3-0b17-4325-8fb5-42df5a883628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:25.954570 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8c9a3-0b17-4325-8fb5-42df5a883628 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:25.955294 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8c9a3-0b17-4325-8fb5-42df5a883628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:25.957539 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8c9a3-0b17-4325-8fb5-42df5a883628 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:25.970739 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:25.986270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc4e4c-e8cd-419b-b850-73d94047ecfd admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:25.993730 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341410b7-fc6d-4463-88cc-8e3cdb3f3402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:25.994186 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 947/1895] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:25 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:25.995192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341410b7-fc6d-4463-88cc-8e3cdb3f3402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:25.995477 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341410b7-fc6d-4463-88cc-8e3cdb3f3402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:25.995765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341410b7-fc6d-4463-88cc-8e3cdb3f3402 admin admin] PATH_INFO: `/v3/projects/1172dfc41b9041d2b884c03390cd372a` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:25.997191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341410b7-fc6d-4463-88cc-8e3cdb3f3402 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1172dfc41b9041d2b884c03390cd372a)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:25.999569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341410b7-fc6d-4463-88cc-8e3cdb3f3402 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:26.007710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8c9a3-0b17-4325-8fb5-42df5a883628 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:26.021029 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8c9a3-0b17-4325-8fb5-42df5a883628 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:26.026602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28461c-dceb-4176-9218-adcf56be379a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:26.026776 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:26.027211 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28461c-dceb-4176-9218-adcf56be379a admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:26.027506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28461c-dceb-4176-9218-adcf56be379a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:26.027788 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28461c-dceb-4176-9218-adcf56be379a admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:26.028902 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf28461c-dceb-4176-9218-adcf56be379a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:26.031429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf28461c-dceb-4176-9218-adcf56be379a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:26.045804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 948/1896] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:26.052176 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28461c-dceb-4176-9218-adcf56be379a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:26.062731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28461c-dceb-4176-9218-adcf56be379a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:26.068263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-341410b7-fc6d-4463-88cc-8e3cdb3f3402 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1172dfc41b9041d2b884c03390cd372a', 'request_id': 'req-341410b7-fc6d-4463-88cc-8e3cdb3f3402'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:26.077697 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-341410b7-fc6d-4463-88cc-8e3cdb3f3402 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1172dfc41b9041d2b884c03390cd372a', 'request_id': 'req-341410b7-fc6d-4463-88cc-8e3cdb3f3402'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:26.088920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-341410b7-fc6d-4463-88cc-8e3cdb3f3402 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1172dfc41b9041d2b884c03390cd372a', 'request_id': 'req-341410b7-fc6d-4463-88cc-8e3cdb3f3402'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:26.102361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-341410b7-fc6d-4463-88cc-8e3cdb3f3402 admin admin] The token cache is being invalidate because project 1172dfc41b9041d2b884c03390cd372a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:26.102704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-341410b7-fc6d-4463-88cc-8e3cdb3f3402 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:26.103414 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-341410b7-fc6d-4463-88cc-8e3cdb3f3402 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:26.105294 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 949/1897] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:25 2022] DELETE /identity/v3/projects/1172dfc41b9041d2b884c03390cd372a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:26.106072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341410b7-fc6d-4463-88cc-8e3cdb3f3402 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:26.107392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-341410b7-fc6d-4463-88cc-8e3cdb3f3402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:26.145922 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ee6aa-886b-40dc-a2d9-faa96ed59da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:26.146603 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ee6aa-886b-40dc-a2d9-faa96ed59da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:26.147098 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ee6aa-886b-40dc-a2d9-faa96ed59da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:26.147383 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ee6aa-886b-40dc-a2d9-faa96ed59da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:26.148120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ee6aa-886b-40dc-a2d9-faa96ed59da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:26.149894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ee6aa-886b-40dc-a2d9-faa96ed59da2 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:26.162158 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 949/1898] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:26.164264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ee6aa-886b-40dc-a2d9-faa96ed59da2 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:26.169057 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-977ee6aa-886b-40dc-a2d9-faa96ed59da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:26.190182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ee6aa-886b-40dc-a2d9-faa96ed59da2 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:26.197904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4feb8-0c21-444a-976b-0bb38a01434b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:26.199182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4feb8-0c21-444a-976b-0bb38a01434b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:26.199466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4feb8-0c21-444a-976b-0bb38a01434b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:26.199747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4feb8-0c21-444a-976b-0bb38a01434b admin admin] PATH_INFO: `/v3/users/4c19880afeae46a1a56e0cf683f44d12` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:26.207406 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4feb8-0c21-444a-976b-0bb38a01434b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c19880afeae46a1a56e0cf683f44d12)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:26.211782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4feb8-0c21-444a-976b-0bb38a01434b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:26.230876 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341410b7-fc6d-4463-88cc-8e3cdb3f3402 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:26.240070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0adaff5-5d92-42d2-a26d-c175d57192ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:26.240982 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0adaff5-5d92-42d2-a26d-c175d57192ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:26.241266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0adaff5-5d92-42d2-a26d-c175d57192ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:26.241545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0adaff5-5d92-42d2-a26d-c175d57192ce admin admin] PATH_INFO: `/v3/users/92017633cf094918bbdad11d0a1f79a5` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:26.243122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0adaff5-5d92-42d2-a26d-c175d57192ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=92017633cf094918bbdad11d0a1f79a5)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:26.245920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0adaff5-5d92-42d2-a26d-c175d57192ce admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:26.279141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-23c4feb8-0c21-444a-976b-0bb38a01434b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c19880afeae46a1a56e0cf683f44d12'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:26.302422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-23c4feb8-0c21-444a-976b-0bb38a01434b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c19880afeae46a1a56e0cf683f44d12'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:26.315366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-23c4feb8-0c21-444a-976b-0bb38a01434b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c19880afeae46a1a56e0cf683f44d12'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:26.316123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-23c4feb8-0c21-444a-976b-0bb38a01434b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c19880afeae46a1a56e0cf683f44d12'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:26.320242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b0adaff5-5d92-42d2-a26d-c175d57192ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92017633cf094918bbdad11d0a1f79a5'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:26.326973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-23c4feb8-0c21-444a-976b-0bb38a01434b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c19880afeae46a1a56e0cf683f44d12'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:26.334503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 950/1899] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:26 2022] DELETE /identity/v3/users/4c19880afeae46a1a56e0cf683f44d12 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:26.338469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4feb8-0c21-444a-976b-0bb38a01434b admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:26.339920 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-23c4feb8-0c21-444a-976b-0bb38a01434b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:26.348018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b0adaff5-5d92-42d2-a26d-c175d57192ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92017633cf094918bbdad11d0a1f79a5'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:26.358227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b0adaff5-5d92-42d2-a26d-c175d57192ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92017633cf094918bbdad11d0a1f79a5'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:26.359186 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b0adaff5-5d92-42d2-a26d-c175d57192ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92017633cf094918bbdad11d0a1f79a5'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:26.368727 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b0adaff5-5d92-42d2-a26d-c175d57192ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92017633cf094918bbdad11d0a1f79a5'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:26.376771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 950/1900] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:26 2022] DELETE /identity/v3/users/92017633cf094918bbdad11d0a1f79a5 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:26.381540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aceaa7-9b64-484a-990b-c0c1dc969062 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:26.381857 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aceaa7-9b64-484a-990b-c0c1dc969062 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:26.382123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aceaa7-9b64-484a-990b-c0c1dc969062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:26.397490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:26.408120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-91aceaa7-9b64-484a-990b-c0c1dc969062 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:26.409403 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-91aceaa7-9b64-484a-990b-c0c1dc969062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:26.419132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4feb8-0c21-444a-976b-0bb38a01434b admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:26.431841 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:26.488466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7283f8-1a8b-4df0-ac7d-96cf539aab71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:26.489106 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 951/1901] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:26.491122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7283f8-1a8b-4df0-ac7d-96cf539aab71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:26.491414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7283f8-1a8b-4df0-ac7d-96cf539aab71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:26.491701 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7283f8-1a8b-4df0-ac7d-96cf539aab71 admin admin] PATH_INFO: `/v3/projects/d6de81c54cc74648a90058199ee3f899/users/06a2c82e1dbf4b508732fd83e74269e7/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:26.502763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aceaa7-9b64-484a-990b-c0c1dc969062 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:26.504784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-91aceaa7-9b64-484a-990b-c0c1dc969062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:26.508169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7283f8-1a8b-4df0-ac7d-96cf539aab71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6de81c54cc74648a90058199ee3f899, user_id=06a2c82e1dbf4b508732fd83e74269e7, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:26.511084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7283f8-1a8b-4df0-ac7d-96cf539aab71 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:26.527809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 951/1902] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:26 2022] PUT /identity/v3/projects/d6de81c54cc74648a90058199ee3f899/users/06a2c82e1dbf4b508732fd83e74269e7/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:26.544888 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ab072-dd8c-4dec-8662-6f7d7ae1e688 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:26.545204 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ab072-dd8c-4dec-8662-6f7d7ae1e688 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:26.545487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ab072-dd8c-4dec-8662-6f7d7ae1e688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:26.598723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-df7ab072-dd8c-4dec-8662-6f7d7ae1e688 None None] MFA Rules not processed for user `06a2c82e1dbf4b508732fd83e74269e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:26.599753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aceaa7-9b64-484a-990b-c0c1dc969062 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:26.599950 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-df7ab072-dd8c-4dec-8662-6f7d7ae1e688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:26.669484 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a51453-f3e1-457c-9c78-6315d1c5f8bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:26.670156 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a51453-f3e1-457c-9c78-6315d1c5f8bd service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:26.670440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a51453-f3e1-457c-9c78-6315d1c5f8bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:26.670782 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a51453-f3e1-457c-9c78-6315d1c5f8bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:26.702320 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 952/1903] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:26.706614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ab072-dd8c-4dec-8662-6f7d7ae1e688 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:26.708300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-df7ab072-dd8c-4dec-8662-6f7d7ae1e688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:26.783531 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a51453-f3e1-457c-9c78-6315d1c5f8bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:26.785591 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a51453-f3e1-457c-9c78-6315d1c5f8bd service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:26.801246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 952/1904] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:26.876641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ab072-dd8c-4dec-8662-6f7d7ae1e688 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:26.884005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91189463-d7ac-4b69-9494-c18c0e22c903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:26.884644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91189463-d7ac-4b69-9494-c18c0e22c903 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:26.884937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91189463-d7ac-4b69-9494-c18c0e22c903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:26.885221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91189463-d7ac-4b69-9494-c18c0e22c903 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:26.886551 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91189463-d7ac-4b69-9494-c18c0e22c903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:26.889473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91189463-d7ac-4b69-9494-c18c0e22c903 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:26.904428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:26.933402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a51453-f3e1-457c-9c78-6315d1c5f8bd service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:26.935910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-55a51453-f3e1-457c-9c78-6315d1c5f8bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:26.939876 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 953/1905] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:26 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:13:26.947952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91189463-d7ac-4b69-9494-c18c0e22c903 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:26.954472 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a51453-f3e1-457c-9c78-6315d1c5f8bd service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:27.058294 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:27.096418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4363af2-a025-43d2-9e58-5f5db259fe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:27.097469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4363af2-a025-43d2-9e58-5f5db259fe5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:27.097851 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4363af2-a025-43d2-9e58-5f5db259fe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:27.098339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4363af2-a025-43d2-9e58-5f5db259fe5a admin admin] PATH_INFO: `/v3/projects/4446e63c2cd145468c4ed36802e5e486` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:27.099869 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4363af2-a025-43d2-9e58-5f5db259fe5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4446e63c2cd145468c4ed36802e5e486)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:27.103002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4363af2-a025-43d2-9e58-5f5db259fe5a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:27.127838 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91189463-d7ac-4b69-9494-c18c0e22c903 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:27.142846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446eea6-2f95-41fb-ac3c-8442586e61a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:27.145378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446eea6-2f95-41fb-ac3c-8442586e61a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:27.146243 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446eea6-2f95-41fb-ac3c-8442586e61a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:27.148366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446eea6-2f95-41fb-ac3c-8442586e61a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:27.149764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446eea6-2f95-41fb-ac3c-8442586e61a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:27.153008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446eea6-2f95-41fb-ac3c-8442586e61a9 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:27.190666 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 954/1906] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:26 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:27.190998 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-a4363af2-a025-43d2-9e58-5f5db259fe5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4446e63c2cd145468c4ed36802e5e486', 'request_id': 'req-a4363af2-a025-43d2-9e58-5f5db259fe5a'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:27.192864 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446eea6-2f95-41fb-ac3c-8442586e61a9 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:27.201519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-a4363af2-a025-43d2-9e58-5f5db259fe5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4446e63c2cd145468c4ed36802e5e486', 'request_id': 'req-a4363af2-a025-43d2-9e58-5f5db259fe5a'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:27.209804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446eea6-2f95-41fb-ac3c-8442586e61a9 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:27.220604 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-a4363af2-a025-43d2-9e58-5f5db259fe5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4446e63c2cd145468c4ed36802e5e486', 'request_id': 'req-a4363af2-a025-43d2-9e58-5f5db259fe5a'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:27.240203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-a4363af2-a025-43d2-9e58-5f5db259fe5a admin admin] The token cache is being invalidate because project 4446e63c2cd145468c4ed36802e5e486 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:27.240545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-a4363af2-a025-43d2-9e58-5f5db259fe5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:27.241855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-a4363af2-a025-43d2-9e58-5f5db259fe5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:27.251270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 953/1907] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:26 2022] DELETE /identity/v3/projects/4446e63c2cd145468c4ed36802e5e486 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:27.255613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd0447-cab6-45af-83c8-6202dd95e85b None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:27.256020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd0447-cab6-45af-83c8-6202dd95e85b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:27.258000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd0447-cab6-45af-83c8-6202dd95e85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:27.301180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-27cd0447-cab6-45af-83c8-6202dd95e85b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:27.304277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-27cd0447-cab6-45af-83c8-6202dd95e85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:27.340105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1dc49d-265c-4e1d-89c4-f5562a39f565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:27.342287 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dc49d-265c-4e1d-89c4-f5562a39f565 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:27.342583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dc49d-265c-4e1d-89c4-f5562a39f565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:27.342884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dc49d-265c-4e1d-89c4-f5562a39f565 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:27.343656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1dc49d-265c-4e1d-89c4-f5562a39f565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:27.350454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1dc49d-265c-4e1d-89c4-f5562a39f565 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:27.391908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 955/1908] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:27.392266 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1dc49d-265c-4e1d-89c4-f5562a39f565 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:27.397576 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8e1dc49d-265c-4e1d-89c4-f5562a39f565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:27.424934 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1dc49d-265c-4e1d-89c4-f5562a39f565 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:27.443232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc219e0e-5717-4396-9c39-dcac241f8bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:27.448293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc219e0e-5717-4396-9c39-dcac241f8bbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:27.448552 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc219e0e-5717-4396-9c39-dcac241f8bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:27.450584 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc219e0e-5717-4396-9c39-dcac241f8bbc admin admin] PATH_INFO: `/v3/projects/3bcdbcfc080f4b09afb2ba8985e0bf86` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:27.453687 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc219e0e-5717-4396-9c39-dcac241f8bbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bcdbcfc080f4b09afb2ba8985e0bf86)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:27.455327 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 954/1909] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:27.460434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc219e0e-5717-4396-9c39-dcac241f8bbc admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:27.465671 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd0447-cab6-45af-83c8-6202dd95e85b None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:27.470410 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-27cd0447-cab6-45af-83c8-6202dd95e85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:27.494739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd0447-cab6-45af-83c8-6202dd95e85b None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:27.516224 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fe544-501a-4bf2-aa27-36f4d0867709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:27.516925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fe544-501a-4bf2-aa27-36f4d0867709 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:27.517601 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fe544-501a-4bf2-aa27-36f4d0867709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:27.517601 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fe544-501a-4bf2-aa27-36f4d0867709 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:27.518506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809fe544-501a-4bf2-aa27-36f4d0867709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:27.519877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809fe544-501a-4bf2-aa27-36f4d0867709 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:27.559591 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 955/1910] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:27.560955 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06491c-9831-4670-94ee-e14ef0b90f4c None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:27.561241 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06491c-9831-4670-94ee-e14ef0b90f4c None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:27.561527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06491c-9831-4670-94ee-e14ef0b90f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:27.609387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-7b06491c-9831-4670-94ee-e14ef0b90f4c None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:27.613568 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-7b06491c-9831-4670-94ee-e14ef0b90f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:27.631619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-bc219e0e-5717-4396-9c39-dcac241f8bbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bcdbcfc080f4b09afb2ba8985e0bf86', 'request_id': 'req-bc219e0e-5717-4396-9c39-dcac241f8bbc'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:27.667609 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-bc219e0e-5717-4396-9c39-dcac241f8bbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bcdbcfc080f4b09afb2ba8985e0bf86', 'request_id': 'req-bc219e0e-5717-4396-9c39-dcac241f8bbc'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:27.707348 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:27.710111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-bc219e0e-5717-4396-9c39-dcac241f8bbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bcdbcfc080f4b09afb2ba8985e0bf86', 'request_id': 'req-bc219e0e-5717-4396-9c39-dcac241f8bbc'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:27.734263 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:27.740718 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-bc219e0e-5717-4396-9c39-dcac241f8bbc admin admin] The token cache is being invalidate because project 3bcdbcfc080f4b09afb2ba8985e0bf86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:27.741162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-bc219e0e-5717-4396-9c39-dcac241f8bbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:27.747242 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-bc219e0e-5717-4396-9c39-dcac241f8bbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:27.747639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 956/1911] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:27.749883 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 956/1912] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:27 2022] DELETE /identity/v3/projects/3bcdbcfc080f4b09afb2ba8985e0bf86 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:27.751223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06491c-9831-4670-94ee-e14ef0b90f4c None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:27.752367 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc219e0e-5717-4396-9c39-dcac241f8bbc admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:27.755901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-bc219e0e-5717-4396-9c39-dcac241f8bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:27.773759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06491c-9831-4670-94ee-e14ef0b90f4c None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:27.794650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150119a2-6cd9-4e5b-9b63-4b7742494fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:27.795772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc219e0e-5717-4396-9c39-dcac241f8bbc admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:27.796778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150119a2-6cd9-4e5b-9b63-4b7742494fab admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:27.797022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150119a2-6cd9-4e5b-9b63-4b7742494fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:27.797305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150119a2-6cd9-4e5b-9b63-4b7742494fab admin admin] PATH_INFO: `/v3/projects/e9bd6686cee94ae5aedc413c4fc6228d/users/3c4c31aa59b54ecda17f76b20cbd1e64/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:27.806384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1d1d3-d8a6-451d-9ca2-7eb6764fb302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:27.813896 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1d1d3-d8a6-451d-9ca2-7eb6764fb302 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:27.814278 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1d1d3-d8a6-451d-9ca2-7eb6764fb302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:27.814566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1d1d3-d8a6-451d-9ca2-7eb6764fb302 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:27.815536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1d1d3-d8a6-451d-9ca2-7eb6764fb302 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:27.815857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1d1d3-d8a6-451d-9ca2-7eb6764fb302 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:27.817500 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1d1d3-d8a6-451d-9ca2-7eb6764fb302 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:27.834343 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150119a2-6cd9-4e5b-9b63-4b7742494fab admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9bd6686cee94ae5aedc413c4fc6228d, user_id=3c4c31aa59b54ecda17f76b20cbd1e64, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:27.841165 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150119a2-6cd9-4e5b-9b63-4b7742494fab admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:27.843507 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 957/1913] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:13:27.849568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1d1d3-d8a6-451d-9ca2-7eb6764fb302 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:27.854082 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-15b1d1d3-d8a6-451d-9ca2-7eb6764fb302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:27.881520 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 957/1914] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:27 2022] PUT /identity/v3/projects/e9bd6686cee94ae5aedc413c4fc6228d/users/3c4c31aa59b54ecda17f76b20cbd1e64/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:27.881520 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150119a2-6cd9-4e5b-9b63-4b7742494fab admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:27.888132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1d1d3-d8a6-451d-9ca2-7eb6764fb302 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:27.991628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbfad6e-c1da-4404-a035-292e87c44c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:27.994003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfad6e-c1da-4404-a035-292e87c44c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:27.995956 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfad6e-c1da-4404-a035-292e87c44c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:27.996390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfad6e-c1da-4404-a035-292e87c44c42 admin admin] PATH_INFO: `/v3/users/1b3f114233064392af6a7e81a676d0b5` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:27.999907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbfad6e-c1da-4404-a035-292e87c44c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b3f114233064392af6a7e81a676d0b5)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:28.005076 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbfad6e-c1da-4404-a035-292e87c44c42 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:28.017524 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150119a2-6cd9-4e5b-9b63-4b7742494fab admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:28.036247 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbfc2d-36fc-4707-a869-457c68107d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:28.037029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbfc2d-36fc-4707-a869-457c68107d82 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:28.037416 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbfc2d-36fc-4707-a869-457c68107d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:28.037806 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbfc2d-36fc-4707-a869-457c68107d82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:28.039061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbfc2d-36fc-4707-a869-457c68107d82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:28.042567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbfc2d-36fc-4707-a869-457c68107d82 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:28.095193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 958/1915] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:27 2022] POST /identity/v3/projects => generated 367 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:28.097635 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b915fe-af6a-4f3e-9b6c-37e446a7b263 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:28.098066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b915fe-af6a-4f3e-9b6c-37e446a7b263 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:28.099036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b915fe-af6a-4f3e-9b6c-37e446a7b263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:28.113490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-acbfad6e-c1da-4404-a035-292e87c44c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b3f114233064392af6a7e81a676d0b5'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:28.137239 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-acbfad6e-c1da-4404-a035-292e87c44c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b3f114233064392af6a7e81a676d0b5'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:28.161984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-acbfad6e-c1da-4404-a035-292e87c44c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b3f114233064392af6a7e81a676d0b5'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:28.165636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-acbfad6e-c1da-4404-a035-292e87c44c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b3f114233064392af6a7e81a676d0b5'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:28.187013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-55b915fe-af6a-4f3e-9b6c-37e446a7b263 None None] MFA Rules not processed for user `3c4c31aa59b54ecda17f76b20cbd1e64`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:28.187221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-acbfad6e-c1da-4404-a035-292e87c44c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b3f114233064392af6a7e81a676d0b5'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:28.189070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-55b915fe-af6a-4f3e-9b6c-37e446a7b263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:28.196186 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 958/1916] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:27 2022] DELETE /identity/v3/users/1b3f114233064392af6a7e81a676d0b5 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:28.198796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fade1d-d4fd-445f-9016-750e56dd931f None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:28.199054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fade1d-d4fd-445f-9016-750e56dd931f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:28.199335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fade1d-d4fd-445f-9016-750e56dd931f None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:28.200758 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 959/1917] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jul 20 09:13:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:13:28.207141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fade1d-d4fd-445f-9016-750e56dd931f None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:28.213568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-83fade1d-d4fd-445f-9016-750e56dd931f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:28.299324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 959/1918] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:28.299324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3843f-c7f0-4900-b5c2-8410888111a6 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:28.299324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3843f-c7f0-4900-b5c2-8410888111a6 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:28.299324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3843f-c7f0-4900-b5c2-8410888111a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:28.329415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-5ee3843f-c7f0-4900-b5c2-8410888111a6 None None] MFA Rules not processed for user `be08c43f0d0c4a8194596e54500b44ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:28.330555 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5ee3843f-c7f0-4900-b5c2-8410888111a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:28.396698 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:28.424406 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fade1d-d4fd-445f-9016-750e56dd931f None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:28.440091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 960/1919] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jul 20 09:13:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:28.440564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0899111-0e31-42bf-92dd-a9774193465b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:28.440812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3843f-c7f0-4900-b5c2-8410888111a6 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:28.441270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0899111-0e31-42bf-92dd-a9774193465b admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:28.441553 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0899111-0e31-42bf-92dd-a9774193465b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:28.441835 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0899111-0e31-42bf-92dd-a9774193465b admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:28.442365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5ee3843f-c7f0-4900-b5c2-8410888111a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:28.442640 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0899111-0e31-42bf-92dd-a9774193465b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:28.447212 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0899111-0e31-42bf-92dd-a9774193465b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:28.468774 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3843f-c7f0-4900-b5c2-8410888111a6 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:28.475401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5166a9-aacf-49d4-9325-b2d81cd9f7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:28.476348 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5166a9-aacf-49d4-9325-b2d81cd9f7be admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:28.476632 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5166a9-aacf-49d4-9325-b2d81cd9f7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:28.476939 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5166a9-aacf-49d4-9325-b2d81cd9f7be admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:28.478447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5166a9-aacf-49d4-9325-b2d81cd9f7be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:28.480882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5166a9-aacf-49d4-9325-b2d81cd9f7be admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:28.487322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:28.507220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 960/1920] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:28 2022] POST /identity/v3/users => generated 284 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:28.509508 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0899111-0e31-42bf-92dd-a9774193465b admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:28.512312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c0899111-0e31-42bf-92dd-a9774193465b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:28.531293 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 961/1921] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:28 2022] POST /identity/v3/projects => generated 387 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:28.532544 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5166a9-aacf-49d4-9325-b2d81cd9f7be admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:28.535806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0899111-0e31-42bf-92dd-a9774193465b admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:28.621624 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5166a9-aacf-49d4-9325-b2d81cd9f7be admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:28.631939 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5d5ae-8cbb-49cd-8ecf-de6f6953a948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:28.632611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5d5ae-8cbb-49cd-8ecf-de6f6953a948 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:28.632901 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5d5ae-8cbb-49cd-8ecf-de6f6953a948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:28.633180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5d5ae-8cbb-49cd-8ecf-de6f6953a948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:28.635453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5d5ae-8cbb-49cd-8ecf-de6f6953a948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:28.636801 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5d5ae-8cbb-49cd-8ecf-de6f6953a948 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:28.647486 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 962/1922] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:28.651273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e922c91-2163-4485-9e14-771e87cd598e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:28.651453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5d5ae-8cbb-49cd-8ecf-de6f6953a948 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:28.655374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e922c91-2163-4485-9e14-771e87cd598e service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:28.656036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e922c91-2163-4485-9e14-771e87cd598e service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:28.656378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e922c91-2163-4485-9e14-771e87cd598e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:28.661615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5d5ae-8cbb-49cd-8ecf-de6f6953a948 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:28.667804 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0922baf-5b8e-48c3-8225-e5892f36cfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:28.669905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0922baf-5b8e-48c3-8225-e5892f36cfa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:28.670298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0922baf-5b8e-48c3-8225-e5892f36cfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:28.670715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0922baf-5b8e-48c3-8225-e5892f36cfa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:28.671522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e922c91-2163-4485-9e14-771e87cd598e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:28.671797 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0922baf-5b8e-48c3-8225-e5892f36cfa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:28.673132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e922c91-2163-4485-9e14-771e87cd598e service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:28.674346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0922baf-5b8e-48c3-8225-e5892f36cfa9 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:28.686784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 961/1923] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:28.687730 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e922c91-2163-4485-9e14-771e87cd598e service nova] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:28.705941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e922c91-2163-4485-9e14-771e87cd598e service nova] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:28.709812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 963/1924] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:28 2022] POST /identity/v3/users => generated 300 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:28.721842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfad42-159a-44c8-90f5-82b89f8f35ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:28.723129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfad42-159a-44c8-90f5-82b89f8f35ac admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:28.723779 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfad42-159a-44c8-90f5-82b89f8f35ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:28.724308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0922baf-5b8e-48c3-8225-e5892f36cfa9 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:28.724475 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cfad42-159a-44c8-90f5-82b89f8f35ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:28.726054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfad42-159a-44c8-90f5-82b89f8f35ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:28.728756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cfad42-159a-44c8-90f5-82b89f8f35ac admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:28.741794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 962/1925] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:28.745761 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0922baf-5b8e-48c3-8225-e5892f36cfa9 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:28.745977 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfad42-159a-44c8-90f5-82b89f8f35ac admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:28.752780 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610aa333-36ea-428c-8cd2-64d5918ad69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:28.754607 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610aa333-36ea-428c-8cd2-64d5918ad69e admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:28.755243 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610aa333-36ea-428c-8cd2-64d5918ad69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:28.755710 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610aa333-36ea-428c-8cd2-64d5918ad69e admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:28.756518 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610aa333-36ea-428c-8cd2-64d5918ad69e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:28.757470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cfad42-159a-44c8-90f5-82b89f8f35ac admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:28.759632 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610aa333-36ea-428c-8cd2-64d5918ad69e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:28.763513 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59902b47-892b-4d3a-b4af-4cae9f088a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:28.764496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59902b47-892b-4d3a-b4af-4cae9f088a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:28.764850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59902b47-892b-4d3a-b4af-4cae9f088a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:28.765197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59902b47-892b-4d3a-b4af-4cae9f088a2d admin admin] PATH_INFO: `/v3/projects/29844ed4364f4bcea590e2db3a69768b/users/61d1f585757248bead507c8c2b18a8e8/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:28.780551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 964/1926] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:28.782132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610aa333-36ea-428c-8cd2-64d5918ad69e admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:28.802306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610aa333-36ea-428c-8cd2-64d5918ad69e admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:28.808405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59902b47-892b-4d3a-b4af-4cae9f088a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=29844ed4364f4bcea590e2db3a69768b, user_id=61d1f585757248bead507c8c2b18a8e8, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:28.810044 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e18037-8c43-400a-a2cf-55460cb583e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:28.811204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e18037-8c43-400a-a2cf-55460cb583e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:28.811558 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e18037-8c43-400a-a2cf-55460cb583e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:28.811903 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e18037-8c43-400a-a2cf-55460cb583e5 admin admin] PATH_INFO: `/v3/projects/de36fd9508b74f948964df55faf7163e/users/828a7185e7c44699a49d976341d15f3a/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:28.817425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59902b47-892b-4d3a-b4af-4cae9f088a2d admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:28.839825 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e18037-8c43-400a-a2cf-55460cb583e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de36fd9508b74f948964df55faf7163e, user_id=828a7185e7c44699a49d976341d15f3a, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:28.842606 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e18037-8c43-400a-a2cf-55460cb583e5 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:28.856195 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 963/1927] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:28 2022] PUT /identity/v3/projects/29844ed4364f4bcea590e2db3a69768b/users/61d1f585757248bead507c8c2b18a8e8/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:28.858318 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f791b16-5fc3-4eb8-ab8d-9608c635518b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:28.858705 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f791b16-5fc3-4eb8-ab8d-9608c635518b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:28.859270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f791b16-5fc3-4eb8-ab8d-9608c635518b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:28.887314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 965/1928] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:28 2022] PUT /identity/v3/projects/de36fd9508b74f948964df55faf7163e/users/828a7185e7c44699a49d976341d15f3a/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:28.891791 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e18037-8c43-400a-a2cf-55460cb583e5 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:28.926317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-5f791b16-5fc3-4eb8-ab8d-9608c635518b None None] MFA Rules not processed for user `61d1f585757248bead507c8c2b18a8e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:28.927643 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5f791b16-5fc3-4eb8-ab8d-9608c635518b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:29.012300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:29.019588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e18037-8c43-400a-a2cf-55460cb583e5 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:29.026543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0eed6-3507-4a57-b9cd-6d97b7f5ce37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:29.027359 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0eed6-3507-4a57-b9cd-6d97b7f5ce37 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.027653 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0eed6-3507-4a57-b9cd-6d97b7f5ce37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.027932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0eed6-3507-4a57-b9cd-6d97b7f5ce37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.028672 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d0eed6-3507-4a57-b9cd-6d97b7f5ce37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:29.030286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d0eed6-3507-4a57-b9cd-6d97b7f5ce37 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:29.037129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 964/1929] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:29.038131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f791b16-5fc3-4eb8-ab8d-9608c635518b None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:29.039621 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5f791b16-5fc3-4eb8-ab8d-9608c635518b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:29.045956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 966/1930] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:29.046927 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0eed6-3507-4a57-b9cd-6d97b7f5ce37 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:29.059471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0eed6-3507-4a57-b9cd-6d97b7f5ce37 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:29.060020 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f791b16-5fc3-4eb8-ab8d-9608c635518b None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:29.066513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c94e6a-466f-44f7-9c25-cbe4bc42da01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:29.066668 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:29.067367 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c94e6a-466f-44f7-9c25-cbe4bc42da01 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.067640 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.067756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c94e6a-466f-44f7-9c25-cbe4bc42da01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.067931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.068071 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c94e6a-466f-44f7-9c25-cbe4bc42da01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.068198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6 admin admin] PATH_INFO: `/v3/projects/50c91168984b4c0f8a127f8ad345befb` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.069542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c94e6a-466f-44f7-9c25-cbe4bc42da01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:29.069649 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50c91168984b4c0f8a127f8ad345befb)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:29.072213 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:29.072340 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c94e6a-466f-44f7-9c25-cbe4bc42da01 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:29.107665 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 967/1931] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:29 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:29.110805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c94e6a-466f-44f7-9c25-cbe4bc42da01 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:29.129045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:29.152500 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50c91168984b4c0f8a127f8ad345befb', 'request_id': 'req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:29.160812 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50c91168984b4c0f8a127f8ad345befb', 'request_id': 'req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:29.176386 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50c91168984b4c0f8a127f8ad345befb', 'request_id': 'req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:29.195153 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6 admin admin] The token cache is being invalidate because project 50c91168984b4c0f8a127f8ad345befb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:29.195453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:29.198654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:29.206402 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 965/1932] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:29 2022] DELETE /identity/v3/projects/50c91168984b4c0f8a127f8ad345befb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:29.207660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:29.209512 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:29.209836 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c94e6a-466f-44f7-9c25-cbe4bc42da01 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:29.314833 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d9d90-0f45-40bd-9df1-a552453a7121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:29.316226 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d9d90-0f45-40bd-9df1-a552453a7121 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.317216 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d9d90-0f45-40bd-9df1-a552453a7121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.318767 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d9d90-0f45-40bd-9df1-a552453a7121 admin admin] PATH_INFO: `/v3/projects/de36fd9508b74f948964df55faf7163e/users/828a7185e7c44699a49d976341d15f3a/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.323437 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d9d90-0f45-40bd-9df1-a552453a7121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de36fd9508b74f948964df55faf7163e, user_id=828a7185e7c44699a49d976341d15f3a, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:29.326985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d9d90-0f45-40bd-9df1-a552453a7121 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:29.344575 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 968/1933] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:29 2022] PUT /identity/v3/projects/de36fd9508b74f948964df55faf7163e/users/828a7185e7c44699a49d976341d15f3a/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:29.346304 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31cddd-3004-407e-a026-5786af430e12 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.346929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31cddd-3004-407e-a026-5786af430e12 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.347357 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31cddd-3004-407e-a026-5786af430e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.361597 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ed818-bb42-4e30-a61b-bfb90ee7f6b6 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:29.394033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-6f31cddd-3004-407e-a026-5786af430e12 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:29.396556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6f31cddd-3004-407e-a026-5786af430e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:29.489391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66cb11-7645-4666-92a3-402e4917900f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:29.490556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66cb11-7645-4666-92a3-402e4917900f admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.490877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66cb11-7645-4666-92a3-402e4917900f admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.491396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66cb11-7645-4666-92a3-402e4917900f admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.492518 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66cb11-7645-4666-92a3-402e4917900f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:29.495669 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66cb11-7645-4666-92a3-402e4917900f admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:29.520225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 969/1934] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:29.528985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bf2c1-0212-4ac7-be32-8d32702b1ea9 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.529465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bf2c1-0212-4ac7-be32-8d32702b1ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.529827 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bf2c1-0212-4ac7-be32-8d32702b1ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.530711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 966/1935] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:29 2022] POST /identity/v3/users => generated 290 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:29.532238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d9c95-b492-4371-bfca-342d31b2a2e2 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.532535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d9c95-b492-4371-bfca-342d31b2a2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.532892 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d9c95-b492-4371-bfca-342d31b2a2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.559797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-c40d9c95-b492-4371-bfca-342d31b2a2e2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:29.561212 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c40d9c95-b492-4371-bfca-342d31b2a2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:29.562850 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:29.573261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 967/1936] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:29.578370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d9c95-b492-4371-bfca-342d31b2a2e2 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:29.579410 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a82bf2c1-0212-4ac7-be32-8d32702b1ea9 None None] MFA Rules not processed for user `828a7185e7c44699a49d976341d15f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:29.582342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a82bf2c1-0212-4ac7-be32-8d32702b1ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:29.590205 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d9c95-b492-4371-bfca-342d31b2a2e2 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:29.596476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97f20a-d9b5-4de0-b96a-f70f729fd2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:29.597242 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97f20a-d9b5-4de0-b96a-f70f729fd2be admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.597607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97f20a-d9b5-4de0-b96a-f70f729fd2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.597970 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb97f20a-d9b5-4de0-b96a-f70f729fd2be admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.599361 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97f20a-d9b5-4de0-b96a-f70f729fd2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:29.601180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb97f20a-d9b5-4de0-b96a-f70f729fd2be admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:29.614561 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 968/1937] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:29.615495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97f20a-d9b5-4de0-b96a-f70f729fd2be admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:29.637021 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb97f20a-d9b5-4de0-b96a-f70f729fd2be admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:29.644772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54725cff-bf16-4dd8-b1d1-9a5a44073827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:29.645749 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54725cff-bf16-4dd8-b1d1-9a5a44073827 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.646041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54725cff-bf16-4dd8-b1d1-9a5a44073827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.646320 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54725cff-bf16-4dd8-b1d1-9a5a44073827 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.647154 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54725cff-bf16-4dd8-b1d1-9a5a44073827 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:29.647954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54725cff-bf16-4dd8-b1d1-9a5a44073827 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:29.651449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54725cff-bf16-4dd8-b1d1-9a5a44073827 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:29.674709 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 969/1938] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:13:29.674709 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54725cff-bf16-4dd8-b1d1-9a5a44073827 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:29.678427 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 970/1939] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:29.679291 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bf2c1-0212-4ac7-be32-8d32702b1ea9 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:29.688466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54725cff-bf16-4dd8-b1d1-9a5a44073827 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:29.695530 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a92b12-9ec8-4b66-abe4-296acd04d4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:29.696892 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a92b12-9ec8-4b66-abe4-296acd04d4f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.697174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a92b12-9ec8-4b66-abe4-296acd04d4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.697454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a92b12-9ec8-4b66-abe4-296acd04d4f0 admin admin] PATH_INFO: `/v3/projects/96bc418392b144d48428d5a5d00c4059/users/69fabc59dc54451e82c52aaf155b4103/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.700877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bf2c1-0212-4ac7-be32-8d32702b1ea9 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:29.707489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974ab36-08c2-43e4-a710-8e271f10c4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:29.708168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974ab36-08c2-43e4-a710-8e271f10c4c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.708454 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974ab36-08c2-43e4-a710-8e271f10c4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.709310 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974ab36-08c2-43e4-a710-8e271f10c4c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.709999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974ab36-08c2-43e4-a710-8e271f10c4c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:29.714381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974ab36-08c2-43e4-a710-8e271f10c4c8 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:29.719873 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a92b12-9ec8-4b66-abe4-296acd04d4f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96bc418392b144d48428d5a5d00c4059, user_id=69fabc59dc54451e82c52aaf155b4103, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:29.722543 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a92b12-9ec8-4b66-abe4-296acd04d4f0 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:29.733558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:29.742794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 970/1940] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:29 2022] PUT /identity/v3/projects/96bc418392b144d48428d5a5d00c4059/users/69fabc59dc54451e82c52aaf155b4103/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:29.742794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a92b12-9ec8-4b66-abe4-296acd04d4f0 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:29.742794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-56a92b12-9ec8-4b66-abe4-296acd04d4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:29.746931 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 971/1941] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:29 2022] POST /identity/v3/projects => generated 399 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:29.748038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974ab36-08c2-43e4-a710-8e271f10c4c8 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:29.804977 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a92b12-9ec8-4b66-abe4-296acd04d4f0 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:29.824387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974ab36-08c2-43e4-a710-8e271f10c4c8 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:29.829534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86154bfc-d13e-4864-88f8-da020ee3862a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:29.830249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86154bfc-d13e-4864-88f8-da020ee3862a admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.830517 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86154bfc-d13e-4864-88f8-da020ee3862a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.830829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86154bfc-d13e-4864-88f8-da020ee3862a admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.831590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86154bfc-d13e-4864-88f8-da020ee3862a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:29.833185 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86154bfc-d13e-4864-88f8-da020ee3862a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:29.843519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 972/1942] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:29.844296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86154bfc-d13e-4864-88f8-da020ee3862a admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:29.853873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86154bfc-d13e-4864-88f8-da020ee3862a admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:29.858622 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e9d58-fbd9-48ff-be09-7fa835d0ab26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:29.858882 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa288223-e12c-4944-9979-4c5541c9929c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:29.859302 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e9d58-fbd9-48ff-be09-7fa835d0ab26 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.859410 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa288223-e12c-4944-9979-4c5541c9929c service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.859586 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e9d58-fbd9-48ff-be09-7fa835d0ab26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.859678 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa288223-e12c-4944-9979-4c5541c9929c service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.859865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e9d58-fbd9-48ff-be09-7fa835d0ab26 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.859957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa288223-e12c-4944-9979-4c5541c9929c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.860539 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e9d58-fbd9-48ff-be09-7fa835d0ab26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:29.862428 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e9d58-fbd9-48ff-be09-7fa835d0ab26 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:29.883054 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 973/1943] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:29 2022] POST /identity/v3/users => generated 300 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:29.884519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e9d58-fbd9-48ff-be09-7fa835d0ab26 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:29.894296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e9d58-fbd9-48ff-be09-7fa835d0ab26 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:29.899804 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b3c56-13c6-48cb-a854-0885b4bb8f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:29.900716 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b3c56-13c6-48cb-a854-0885b4bb8f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.901132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b3c56-13c6-48cb-a854-0885b4bb8f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.901421 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b3c56-13c6-48cb-a854-0885b4bb8f43 admin admin] PATH_INFO: `/v3/projects/96bc418392b144d48428d5a5d00c4059/users/69fabc59dc54451e82c52aaf155b4103/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.903447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b3c56-13c6-48cb-a854-0885b4bb8f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96bc418392b144d48428d5a5d00c4059, user_id=69fabc59dc54451e82c52aaf155b4103, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:29.906080 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b3c56-13c6-48cb-a854-0885b4bb8f43 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:29.917476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 974/1944] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:29 2022] PUT /identity/v3/projects/96bc418392b144d48428d5a5d00c4059/users/69fabc59dc54451e82c52aaf155b4103/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:29.918337 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b3c56-13c6-48cb-a854-0885b4bb8f43 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:29.923832 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa288223-e12c-4944-9979-4c5541c9929c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:29.925261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa288223-e12c-4944-9979-4c5541c9929c service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:29.935647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 971/1945] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:29.937510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d035b-a34b-40a8-8f20-991c1f51974f None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:29.937804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d035b-a34b-40a8-8f20-991c1f51974f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:29.938082 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d035b-a34b-40a8-8f20-991c1f51974f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:29.965935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-243d035b-a34b-40a8-8f20-991c1f51974f None None] MFA Rules not processed for user `69fabc59dc54451e82c52aaf155b4103`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:29.967199 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-243d035b-a34b-40a8-8f20-991c1f51974f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:29.974799 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:30.007821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b3c56-13c6-48cb-a854-0885b4bb8f43 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:30.012892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74baec7e-e660-4a44-9404-8c3d4543ae90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:30.013571 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baec7e-e660-4a44-9404-8c3d4543ae90 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:30.013869 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baec7e-e660-4a44-9404-8c3d4543ae90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:30.014144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baec7e-e660-4a44-9404-8c3d4543ae90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:30.015151 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74baec7e-e660-4a44-9404-8c3d4543ae90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:30.016799 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74baec7e-e660-4a44-9404-8c3d4543ae90 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:30.025226 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:30.033650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 975/1946] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:30.034922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74baec7e-e660-4a44-9404-8c3d4543ae90 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:30.043314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 972/1947] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:29 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:30.046986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74baec7e-e660-4a44-9404-8c3d4543ae90 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:30.051830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfe98b-09c3-4fc8-bb2b-1e88b3d968d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:30.052493 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfe98b-09c3-4fc8-bb2b-1e88b3d968d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:30.052772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfe98b-09c3-4fc8-bb2b-1e88b3d968d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:30.053051 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfe98b-09c3-4fc8-bb2b-1e88b3d968d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:30.053777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcfe98b-09c3-4fc8-bb2b-1e88b3d968d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:30.055455 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcfe98b-09c3-4fc8-bb2b-1e88b3d968d9 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:30.057226 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d035b-a34b-40a8-8f20-991c1f51974f None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:30.058670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-243d035b-a34b-40a8-8f20-991c1f51974f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:30.066350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 976/1948] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:30.070637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfe98b-09c3-4fc8-bb2b-1e88b3d968d9 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:30.083159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfe98b-09c3-4fc8-bb2b-1e88b3d968d9 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:30.091263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e55019-2c74-4df7-98a7-499197a3f92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:30.092917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e55019-2c74-4df7-98a7-499197a3f92d admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:30.093376 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e55019-2c74-4df7-98a7-499197a3f92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:30.093801 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e55019-2c74-4df7-98a7-499197a3f92d admin admin] PATH_INFO: `/v3/projects/199037e62a2e4e49bdf5363562383ad6/users/ad0bb41e5bda40d98469dc1a83d50073/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:30.112752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e55019-2c74-4df7-98a7-499197a3f92d admin admin] RBAC: Authorizing `identity:create_grant(project_id=199037e62a2e4e49bdf5363562383ad6, user_id=ad0bb41e5bda40d98469dc1a83d50073, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:30.115544 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e55019-2c74-4df7-98a7-499197a3f92d admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:30.124937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d035b-a34b-40a8-8f20-991c1f51974f None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:30.132301 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 977/1949] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:30 2022] PUT /identity/v3/projects/199037e62a2e4e49bdf5363562383ad6/users/ad0bb41e5bda40d98469dc1a83d50073/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:30.143218 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60f524-f2fc-4738-ac64-29791cdc0d14 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:30.143995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60f524-f2fc-4738-ac64-29791cdc0d14 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:30.143995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60f524-f2fc-4738-ac64-29791cdc0d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:30.172811 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-fe60f524-f2fc-4738-ac64-29791cdc0d14 None None] MFA Rules not processed for user `ad0bb41e5bda40d98469dc1a83d50073`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:30.174145 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-fe60f524-f2fc-4738-ac64-29791cdc0d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:30.179947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29ff40-4972-43f7-b716-fbfe1d66fcd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:30.181170 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29ff40-4972-43f7-b716-fbfe1d66fcd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:30.181437 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29ff40-4972-43f7-b716-fbfe1d66fcd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:30.181722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29ff40-4972-43f7-b716-fbfe1d66fcd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:30.259786 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29ff40-4972-43f7-b716-fbfe1d66fcd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:30.262304 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29ff40-4972-43f7-b716-fbfe1d66fcd5 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:30.275834 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 973/1950] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:30 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:30.292515 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 978/1951] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:30.296995 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29ff40-4972-43f7-b716-fbfe1d66fcd5 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:30.395797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:30.413338 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29ff40-4972-43f7-b716-fbfe1d66fcd5 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:30.420389 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f877df-0a2f-4f14-a985-0e7ab83c2f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:30.421015 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f877df-0a2f-4f14-a985-0e7ab83c2f5e admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:30.421305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f877df-0a2f-4f14-a985-0e7ab83c2f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:30.421593 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f877df-0a2f-4f14-a985-0e7ab83c2f5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:30.422362 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f877df-0a2f-4f14-a985-0e7ab83c2f5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:30.425085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60f524-f2fc-4738-ac64-29791cdc0d14 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:30.426334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f877df-0a2f-4f14-a985-0e7ab83c2f5e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:30.436209 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60f524-f2fc-4738-ac64-29791cdc0d14 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:30.441923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79bcf4-b409-4459-b372-774ab672808c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:30.443222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79bcf4-b409-4459-b372-774ab672808c service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:30.443712 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79bcf4-b409-4459-b372-774ab672808c service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:30.444188 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79bcf4-b409-4459-b372-774ab672808c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:30.448648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-da79bcf4-b409-4459-b372-774ab672808c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:30.470960 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 974/1952] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:30 2022] POST /identity/v3/projects => generated 399 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:30.478743 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f877df-0a2f-4f14-a985-0e7ab83c2f5e admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:30.556095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f877df-0a2f-4f14-a985-0e7ab83c2f5e admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:30.564606 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba49cb-f333-4960-900b-db7fb6268e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:30.565590 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba49cb-f333-4960-900b-db7fb6268e55 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:30.565951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba49cb-f333-4960-900b-db7fb6268e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:30.566307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dba49cb-f333-4960-900b-db7fb6268e55 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:30.567121 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba49cb-f333-4960-900b-db7fb6268e55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:30.569142 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dba49cb-f333-4960-900b-db7fb6268e55 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:30.575361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79bcf4-b409-4459-b372-774ab672808c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:30.576904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79bcf4-b409-4459-b372-774ab672808c service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:30.587451 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 979/1953] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:30.601332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 975/1954] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:30 2022] POST /identity/v3/users => generated 306 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:30.606255 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79bcf4-b409-4459-b372-774ab672808c service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:30.613554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba49cb-f333-4960-900b-db7fb6268e55 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:30.620491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79bcf4-b409-4459-b372-774ab672808c service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:30.632152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf92f0d-0659-4487-a577-e5884cf0c5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:30.632882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf92f0d-0659-4487-a577-e5884cf0c5c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:30.633170 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf92f0d-0659-4487-a577-e5884cf0c5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:30.634797 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf92f0d-0659-4487-a577-e5884cf0c5c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:30.635595 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf92f0d-0659-4487-a577-e5884cf0c5c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:30.637941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf92f0d-0659-4487-a577-e5884cf0c5c6 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:30.651436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 980/1955] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:30.656413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf92f0d-0659-4487-a577-e5884cf0c5c6 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:30.672558 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf92f0d-0659-4487-a577-e5884cf0c5c6 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:30.677665 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99035c-a62d-4906-bc97-9a43b816f54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:30.678565 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99035c-a62d-4906-bc97-9a43b816f54c admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:30.678939 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99035c-a62d-4906-bc97-9a43b816f54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:30.679218 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99035c-a62d-4906-bc97-9a43b816f54c admin admin] PATH_INFO: `/v3/projects/aa6ede4bbe244f068ae6e5d88bda7adf/users/e2513cacb68643dea8991a67a52ee0ec/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:30.696610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99035c-a62d-4906-bc97-9a43b816f54c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa6ede4bbe244f068ae6e5d88bda7adf, user_id=e2513cacb68643dea8991a67a52ee0ec, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:30.697990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dba49cb-f333-4960-900b-db7fb6268e55 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:30.699466 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a99035c-a62d-4906-bc97-9a43b816f54c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:30.701833 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:30.703281 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73e963-0233-4a7e-9939-30ed663252b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:30.704044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73e963-0233-4a7e-9939-30ed663252b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:30.704330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73e963-0233-4a7e-9939-30ed663252b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:30.704621 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73e963-0233-4a7e-9939-30ed663252b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:30.706589 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5d73e963-0233-4a7e-9939-30ed663252b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:30.718213 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 981/1956] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:30 2022] PUT /identity/v3/projects/aa6ede4bbe244f068ae6e5d88bda7adf/users/e2513cacb68643dea8991a67a52ee0ec/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:30.722316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99035c-a62d-4906-bc97-9a43b816f54c admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:30.773628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73e963-0233-4a7e-9939-30ed663252b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:30.775264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73e963-0233-4a7e-9939-30ed663252b7 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:30.785450 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 976/1957] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:30.794832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99035c-a62d-4906-bc97-9a43b816f54c admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:30.799614 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb9fea-99a2-421a-bbad-166e24c8f619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:30.800286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb9fea-99a2-421a-bbad-166e24c8f619 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:30.800573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb9fea-99a2-421a-bbad-166e24c8f619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:30.800854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb9fea-99a2-421a-bbad-166e24c8f619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:30.801593 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb9fea-99a2-421a-bbad-166e24c8f619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:30.803223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb9fea-99a2-421a-bbad-166e24c8f619 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:30.813837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 982/1958] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:30.818228 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73e963-0233-4a7e-9939-30ed663252b7 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:30.830405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73e963-0233-4a7e-9939-30ed663252b7 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:30.838290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16797f9e-eee2-49eb-aa93-cb1f3312d016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:30.839629 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16797f9e-eee2-49eb-aa93-cb1f3312d016 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:30.839917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16797f9e-eee2-49eb-aa93-cb1f3312d016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:30.840332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16797f9e-eee2-49eb-aa93-cb1f3312d016 admin admin] PATH_INFO: `/v3/projects/aa6ede4bbe244f068ae6e5d88bda7adf/users/e2513cacb68643dea8991a67a52ee0ec/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:30.843282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16797f9e-eee2-49eb-aa93-cb1f3312d016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa6ede4bbe244f068ae6e5d88bda7adf, user_id=e2513cacb68643dea8991a67a52ee0ec, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:30.847054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16797f9e-eee2-49eb-aa93-cb1f3312d016 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:30.864891 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 977/1959] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:30 2022] PUT /identity/v3/projects/aa6ede4bbe244f068ae6e5d88bda7adf/users/e2513cacb68643dea8991a67a52ee0ec/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:30.877750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbf3f8-44ee-42d1-bdb7-9d8dce04fafa None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:30.878045 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbf3f8-44ee-42d1-bdb7-9d8dce04fafa None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:30.878360 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbf3f8-44ee-42d1-bdb7-9d8dce04fafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:30.917690 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-d8dbf3f8-44ee-42d1-bdb7-9d8dce04fafa None None] MFA Rules not processed for user `e2513cacb68643dea8991a67a52ee0ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:30.920121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d8dbf3f8-44ee-42d1-bdb7-9d8dce04fafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:31.037022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 983/1960] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:30 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:31.057142 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16797f9e-eee2-49eb-aa93-cb1f3312d016 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:31.072449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:31.121989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16797f9e-eee2-49eb-aa93-cb1f3312d016 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:31.125906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb879af-1da5-43ae-bf70-2323d7f59b95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:31.126361 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb879af-1da5-43ae-bf70-2323d7f59b95 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:31.126648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb879af-1da5-43ae-bf70-2323d7f59b95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:31.126964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb879af-1da5-43ae-bf70-2323d7f59b95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:31.136613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb879af-1da5-43ae-bf70-2323d7f59b95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:31.138113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb879af-1da5-43ae-bf70-2323d7f59b95 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:31.154008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 978/1961] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:31 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:31.225335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbf3f8-44ee-42d1-bdb7-9d8dce04fafa None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:31.235657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbf3f8-44ee-42d1-bdb7-9d8dce04fafa None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:31.239147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0d4db-56df-40d4-a691-6c2e7d7ecc9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:31.239782 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0d4db-56df-40d4-a691-6c2e7d7ecc9b service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:31.240063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0d4db-56df-40d4-a691-6c2e7d7ecc9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:31.240336 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0d4db-56df-40d4-a691-6c2e7d7ecc9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:31.295686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0d4db-56df-40d4-a691-6c2e7d7ecc9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:31.297129 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0d4db-56df-40d4-a691-6c2e7d7ecc9b service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:31.306018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 984/1962] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:31 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:35.592706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb879af-1da5-43ae-bf70-2323d7f59b95 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:35.596651 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5cb879af-1da5-43ae-bf70-2323d7f59b95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:35.738449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb879af-1da5-43ae-bf70-2323d7f59b95 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:35.746907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e07827-6a40-419d-a3eb-70301d39e8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:35.747903 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e07827-6a40-419d-a3eb-70301d39e8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:35.749095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e07827-6a40-419d-a3eb-70301d39e8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:35.749382 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e07827-6a40-419d-a3eb-70301d39e8d7 admin admin] PATH_INFO: `/v3/users/6895a002562c4cf0b13ddfe4e5dbf26d` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:35.750817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e07827-6a40-419d-a3eb-70301d39e8d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6895a002562c4cf0b13ddfe4e5dbf26d)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:35.754049 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e07827-6a40-419d-a3eb-70301d39e8d7 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:35.837818 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d8e07827-6a40-419d-a3eb-70301d39e8d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6895a002562c4cf0b13ddfe4e5dbf26d'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:35.852789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d8e07827-6a40-419d-a3eb-70301d39e8d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6895a002562c4cf0b13ddfe4e5dbf26d'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:35.859876 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d8e07827-6a40-419d-a3eb-70301d39e8d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6895a002562c4cf0b13ddfe4e5dbf26d'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:35.862324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d8e07827-6a40-419d-a3eb-70301d39e8d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6895a002562c4cf0b13ddfe4e5dbf26d'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:35.867321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0d4db-56df-40d4-a691-6c2e7d7ecc9b service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:35.875323 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d8e07827-6a40-419d-a3eb-70301d39e8d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6895a002562c4cf0b13ddfe4e5dbf26d'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:35.881392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0d4db-56df-40d4-a691-6c2e7d7ecc9b service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:35.886821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 979/1963] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:35 2022] DELETE /identity/v3/users/6895a002562c4cf0b13ddfe4e5dbf26d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:35.888918 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f65e5-2e38-4d6f-bf73-9aae3cc340a3 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:35.889069 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f65e5-2e38-4d6f-bf73-9aae3cc340a3 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:35.889349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f65e5-2e38-4d6f-bf73-9aae3cc340a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:35.895927 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:35.923075 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-d30f65e5-2e38-4d6f-bf73-9aae3cc340a3 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:35.924836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d30f65e5-2e38-4d6f-bf73-9aae3cc340a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:35.961562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:35.976434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834f2b3-0bd6-4060-aa2a-9bfcddb53c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:35.978960 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834f2b3-0bd6-4060-aa2a-9bfcddb53c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:35.979568 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834f2b3-0bd6-4060-aa2a-9bfcddb53c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:35.979993 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834f2b3-0bd6-4060-aa2a-9bfcddb53c46 admin admin] PATH_INFO: `/v3/users/ad0bb41e5bda40d98469dc1a83d50073` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:35.981767 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834f2b3-0bd6-4060-aa2a-9bfcddb53c46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad0bb41e5bda40d98469dc1a83d50073)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:35.985241 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834f2b3-0bd6-4060-aa2a-9bfcddb53c46 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:36.018144 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 980/1964] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:36.092803 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f65e5-2e38-4d6f-bf73-9aae3cc340a3 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:36.096375 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d30f65e5-2e38-4d6f-bf73-9aae3cc340a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:36.096645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2834f2b3-0bd6-4060-aa2a-9bfcddb53c46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad0bb41e5bda40d98469dc1a83d50073'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:36.144860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2834f2b3-0bd6-4060-aa2a-9bfcddb53c46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad0bb41e5bda40d98469dc1a83d50073'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:36.171676 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2834f2b3-0bd6-4060-aa2a-9bfcddb53c46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad0bb41e5bda40d98469dc1a83d50073'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:36.172946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2834f2b3-0bd6-4060-aa2a-9bfcddb53c46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad0bb41e5bda40d98469dc1a83d50073'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:36.178999 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f65e5-2e38-4d6f-bf73-9aae3cc340a3 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:36.183773 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcb76b-10c9-442e-81d2-360ea37bf96c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:36.184424 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcb76b-10c9-442e-81d2-360ea37bf96c service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:36.184706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcb76b-10c9-442e-81d2-360ea37bf96c service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:36.184982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcb76b-10c9-442e-81d2-360ea37bf96c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:36.207780 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfcb76b-10c9-442e-81d2-360ea37bf96c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:36.209777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfcb76b-10c9-442e-81d2-360ea37bf96c service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:36.216364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2834f2b3-0bd6-4060-aa2a-9bfcddb53c46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad0bb41e5bda40d98469dc1a83d50073'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:36.225027 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 981/1965] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:36.225747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 985/1966] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:35 2022] DELETE /identity/v3/users/ad0bb41e5bda40d98469dc1a83d50073 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:36.236634 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47babee-c06e-4fc4-9da3-69a215efb3c3 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:36.237396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47babee-c06e-4fc4-9da3-69a215efb3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:36.237841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47babee-c06e-4fc4-9da3-69a215efb3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:36.258902 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-a47babee-c06e-4fc4-9da3-69a215efb3c3 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:36.259990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-a47babee-c06e-4fc4-9da3-69a215efb3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:36.349193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 982/1967] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:36.374180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834f2b3-0bd6-4060-aa2a-9bfcddb53c46 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:36.375834 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-2834f2b3-0bd6-4060-aa2a-9bfcddb53c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:36.417843 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47babee-c06e-4fc4-9da3-69a215efb3c3 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:36.419946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-a47babee-c06e-4fc4-9da3-69a215efb3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:36.496886 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834f2b3-0bd6-4060-aa2a-9bfcddb53c46 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:36.504015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3718a-5c64-4213-9489-11fc3016b1db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:36.504791 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3718a-5c64-4213-9489-11fc3016b1db service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:36.505082 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3718a-5c64-4213-9489-11fc3016b1db service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:36.506044 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3718a-5c64-4213-9489-11fc3016b1db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:36.515659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47babee-c06e-4fc4-9da3-69a215efb3c3 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:36.523790 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f3718a-5c64-4213-9489-11fc3016b1db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:36.525628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f3718a-5c64-4213-9489-11fc3016b1db service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:36.526752 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e1a31-ffe9-4ce3-aef5-14a9437fffff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:36.527444 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e1a31-ffe9-4ce3-aef5-14a9437fffff service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:36.527732 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e1a31-ffe9-4ce3-aef5-14a9437fffff service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:36.532655 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e1a31-ffe9-4ce3-aef5-14a9437fffff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:36.536621 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b82e1a31-ffe9-4ce3-aef5-14a9437fffff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:36.544474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 986/1968] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:36.567103 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3718a-5c64-4213-9489-11fc3016b1db service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:36.568548 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-22f3718a-5c64-4213-9489-11fc3016b1db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:36.594013 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:36.594776 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3718a-5c64-4213-9489-11fc3016b1db service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:36.608445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a354e-e19c-45aa-ac49-f5151b12b2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:36.609542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a354e-e19c-45aa-ac49-f5151b12b2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:36.609821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a354e-e19c-45aa-ac49-f5151b12b2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:36.610105 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418a354e-e19c-45aa-ac49-f5151b12b2da admin admin] PATH_INFO: `/v3/projects/8276462b52db454c8a859cba0096d475` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:36.611753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a354e-e19c-45aa-ac49-f5151b12b2da admin admin] RBAC: Authorizing `identity:delete_project(project_id=8276462b52db454c8a859cba0096d475)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:36.615904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418a354e-e19c-45aa-ac49-f5151b12b2da admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:36.652248 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e1a31-ffe9-4ce3-aef5-14a9437fffff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:36.654543 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e1a31-ffe9-4ce3-aef5-14a9437fffff service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:36.671679 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 983/1969] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:36.743076 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:36.775723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-418a354e-e19c-45aa-ac49-f5151b12b2da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8276462b52db454c8a859cba0096d475', 'request_id': 'req-418a354e-e19c-45aa-ac49-f5151b12b2da'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:36.792107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-418a354e-e19c-45aa-ac49-f5151b12b2da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8276462b52db454c8a859cba0096d475', 'request_id': 'req-418a354e-e19c-45aa-ac49-f5151b12b2da'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:36.813759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-418a354e-e19c-45aa-ac49-f5151b12b2da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8276462b52db454c8a859cba0096d475', 'request_id': 'req-418a354e-e19c-45aa-ac49-f5151b12b2da'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:36.839391 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-418a354e-e19c-45aa-ac49-f5151b12b2da admin admin] The token cache is being invalidate because project 8276462b52db454c8a859cba0096d475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:36.839690 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-418a354e-e19c-45aa-ac49-f5151b12b2da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:36.842153 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-418a354e-e19c-45aa-ac49-f5151b12b2da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:36.845726 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 987/1970] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:36 2022] DELETE /identity/v3/projects/8276462b52db454c8a859cba0096d475 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:36.849833 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e1a31-ffe9-4ce3-aef5-14a9437fffff service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:36.871724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e1a31-ffe9-4ce3-aef5-14a9437fffff service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:37.021943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8ba65-2f90-4db3-8686-a72e574cd722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:37.023039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8ba65-2f90-4db3-8686-a72e574cd722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:37.023485 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8ba65-2f90-4db3-8686-a72e574cd722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:37.023857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8ba65-2f90-4db3-8686-a72e574cd722 admin admin] PATH_INFO: `/v3/users/80457bae80ab410cbef5d1e988270046` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:37.025797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8ba65-2f90-4db3-8686-a72e574cd722 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80457bae80ab410cbef5d1e988270046)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:37.030946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8ba65-2f90-4db3-8686-a72e574cd722 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:37.109101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5da8ba65-2f90-4db3-8686-a72e574cd722 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80457bae80ab410cbef5d1e988270046'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:37.131429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5da8ba65-2f90-4db3-8686-a72e574cd722 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80457bae80ab410cbef5d1e988270046'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:37.143700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5da8ba65-2f90-4db3-8686-a72e574cd722 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80457bae80ab410cbef5d1e988270046'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:37.146069 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5da8ba65-2f90-4db3-8686-a72e574cd722 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80457bae80ab410cbef5d1e988270046'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:37.163667 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5da8ba65-2f90-4db3-8686-a72e574cd722 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80457bae80ab410cbef5d1e988270046'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:37.176710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 984/1971] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:36 2022] DELETE /identity/v3/users/80457bae80ab410cbef5d1e988270046 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:37.391932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a354e-e19c-45aa-ac49-f5151b12b2da admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:37.394939 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-418a354e-e19c-45aa-ac49-f5151b12b2da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:37.416430 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418a354e-e19c-45aa-ac49-f5151b12b2da admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:37.531171 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697b4d0-9537-43a3-91d6-4210d61d954b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:37.532572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697b4d0-9537-43a3-91d6-4210d61d954b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:37.533740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697b4d0-9537-43a3-91d6-4210d61d954b admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:37.534897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697b4d0-9537-43a3-91d6-4210d61d954b admin admin] PATH_INFO: `/v3/users/61d1f585757248bead507c8c2b18a8e8` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:37.542785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697b4d0-9537-43a3-91d6-4210d61d954b admin admin] RBAC: Authorizing `identity:delete_user(user_id=61d1f585757248bead507c8c2b18a8e8)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:37.546745 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697b4d0-9537-43a3-91d6-4210d61d954b admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:37.650218 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8697b4d0-9537-43a3-91d6-4210d61d954b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61d1f585757248bead507c8c2b18a8e8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:37.664882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8697b4d0-9537-43a3-91d6-4210d61d954b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61d1f585757248bead507c8c2b18a8e8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:37.674311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8697b4d0-9537-43a3-91d6-4210d61d954b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61d1f585757248bead507c8c2b18a8e8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:37.675287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8697b4d0-9537-43a3-91d6-4210d61d954b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61d1f585757248bead507c8c2b18a8e8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:37.691515 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8697b4d0-9537-43a3-91d6-4210d61d954b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61d1f585757248bead507c8c2b18a8e8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:37.700321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 988/1972] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:37 2022] DELETE /identity/v3/users/61d1f585757248bead507c8c2b18a8e8 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:37.705775 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8e03e-eb6a-4d26-aab7-26e459ed22a2 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:37.706623 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8e03e-eb6a-4d26-aab7-26e459ed22a2 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:37.707475 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8e03e-eb6a-4d26-aab7-26e459ed22a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:37.732301 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697b4d0-9537-43a3-91d6-4210d61d954b admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:37.735827 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8697b4d0-9537-43a3-91d6-4210d61d954b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:37.759092 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-b7d8e03e-eb6a-4d26-aab7-26e459ed22a2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:37.760974 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b7d8e03e-eb6a-4d26-aab7-26e459ed22a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:37.840265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:37.857727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:37.882261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 985/1973] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:37.883243 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8e03e-eb6a-4d26-aab7-26e459ed22a2 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:37.893118 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b7d8e03e-eb6a-4d26-aab7-26e459ed22a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:37.913391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8e03e-eb6a-4d26-aab7-26e459ed22a2 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:37.920806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:37.921690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:37.921980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:37.922254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2 admin admin] PATH_INFO: `/v3/projects/ad6278c315994028a3fc637d826a291d` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:37.923521 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad6278c315994028a3fc637d826a291d)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:37.925538 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:37.942941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697b4d0-9537-43a3-91d6-4210d61d954b admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:37.951316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f9781-ffa9-4ca8-96de-d291764b871a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:37.952256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f9781-ffa9-4ca8-96de-d291764b871a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:37.952546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f9781-ffa9-4ca8-96de-d291764b871a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:37.952826 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f9781-ffa9-4ca8-96de-d291764b871a admin admin] PATH_INFO: `/v3/projects/199037e62a2e4e49bdf5363562383ad6` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:37.954138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f9781-ffa9-4ca8-96de-d291764b871a admin admin] RBAC: Authorizing `identity:delete_project(project_id=199037e62a2e4e49bdf5363562383ad6)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:37.956142 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f9781-ffa9-4ca8-96de-d291764b871a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:38.038184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad6278c315994028a3fc637d826a291d', 'request_id': 'req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.043738 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-5d0f9781-ffa9-4ca8-96de-d291764b871a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '199037e62a2e4e49bdf5363562383ad6', 'request_id': 'req-5d0f9781-ffa9-4ca8-96de-d291764b871a'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.046960 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad6278c315994028a3fc637d826a291d', 'request_id': 'req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.054933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad6278c315994028a3fc637d826a291d', 'request_id': 'req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.056273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-5d0f9781-ffa9-4ca8-96de-d291764b871a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '199037e62a2e4e49bdf5363562383ad6', 'request_id': 'req-5d0f9781-ffa9-4ca8-96de-d291764b871a'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.067166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-5d0f9781-ffa9-4ca8-96de-d291764b871a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '199037e62a2e4e49bdf5363562383ad6', 'request_id': 'req-5d0f9781-ffa9-4ca8-96de-d291764b871a'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.069692 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2 admin admin] The token cache is being invalidate because project ad6278c315994028a3fc637d826a291d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:38.071070 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.071619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.073955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 986/1974] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:37 2022] DELETE /identity/v3/projects/ad6278c315994028a3fc637d826a291d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:38.074789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:38.078522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:38.086749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-5d0f9781-ffa9-4ca8-96de-d291764b871a admin admin] The token cache is being invalidate because project 199037e62a2e4e49bdf5363562383ad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:38.087629 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-5d0f9781-ffa9-4ca8-96de-d291764b871a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.087896 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-5d0f9781-ffa9-4ca8-96de-d291764b871a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.089542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 989/1975] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:37 2022] DELETE /identity/v3/projects/199037e62a2e4e49bdf5363562383ad6 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:38.093521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aafab5-0355-4a82-8dd9-8fc551b2e92c None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:38.093815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aafab5-0355-4a82-8dd9-8fc551b2e92c None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:38.094096 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aafab5-0355-4a82-8dd9-8fc551b2e92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:38.114222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-28aafab5-0355-4a82-8dd9-8fc551b2e92c None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:38.115463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-28aafab5-0355-4a82-8dd9-8fc551b2e92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:38.147486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5e785-9ab5-42ee-aa2a-f9a34e6808c2 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:38.151813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a32b9e-b69e-4e7b-8628-0d03d803d2c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:38.152782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a32b9e-b69e-4e7b-8628-0d03d803d2c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:38.153062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a32b9e-b69e-4e7b-8628-0d03d803d2c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:38.154082 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a32b9e-b69e-4e7b-8628-0d03d803d2c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:38.155281 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-a0a32b9e-b69e-4e7b-8628-0d03d803d2c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:38.185599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 990/1976] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:38.186985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aafab5-0355-4a82-8dd9-8fc551b2e92c None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:38.188437 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-28aafab5-0355-4a82-8dd9-8fc551b2e92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:38.202817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aafab5-0355-4a82-8dd9-8fc551b2e92c None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:38.208162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234421b-a7b8-4ad5-84d8-1b9e1b6e75a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:38.209092 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234421b-a7b8-4ad5-84d8-1b9e1b6e75a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:38.209379 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234421b-a7b8-4ad5-84d8-1b9e1b6e75a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:38.209656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234421b-a7b8-4ad5-84d8-1b9e1b6e75a2 admin admin] PATH_INFO: `/v3/users/e2513cacb68643dea8991a67a52ee0ec` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:38.211103 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234421b-a7b8-4ad5-84d8-1b9e1b6e75a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2513cacb68643dea8991a67a52ee0ec)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:38.213417 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234421b-a7b8-4ad5-84d8-1b9e1b6e75a2 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:38.230393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a32b9e-b69e-4e7b-8628-0d03d803d2c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:38.232210 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a32b9e-b69e-4e7b-8628-0d03d803d2c1 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:38.243852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 987/1977] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:38.256297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5dafd-4065-4d4d-8846-f9f598fea018 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:38.256589 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5dafd-4065-4d4d-8846-f9f598fea018 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:38.256919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5dafd-4065-4d4d-8846-f9f598fea018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:38.272447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-c234421b-a7b8-4ad5-84d8-1b9e1b6e75a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2513cacb68643dea8991a67a52ee0ec'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.278881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-7dc5dafd-4065-4d4d-8846-f9f598fea018 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:38.280437 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7dc5dafd-4065-4d4d-8846-f9f598fea018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:38.285700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:38.287268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-c234421b-a7b8-4ad5-84d8-1b9e1b6e75a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2513cacb68643dea8991a67a52ee0ec'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.294384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 988/1978] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:38.296820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5dafd-4065-4d4d-8846-f9f598fea018 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:38.302314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-c234421b-a7b8-4ad5-84d8-1b9e1b6e75a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2513cacb68643dea8991a67a52ee0ec'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.303716 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-c234421b-a7b8-4ad5-84d8-1b9e1b6e75a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2513cacb68643dea8991a67a52ee0ec'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.310953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5dafd-4065-4d4d-8846-f9f598fea018 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:38.317406 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-c234421b-a7b8-4ad5-84d8-1b9e1b6e75a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2513cacb68643dea8991a67a52ee0ec'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.318622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a358dd-9577-493d-8068-3517bba020ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:38.319445 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a358dd-9577-493d-8068-3517bba020ea admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:38.319729 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a358dd-9577-493d-8068-3517bba020ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:38.320046 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a358dd-9577-493d-8068-3517bba020ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:38.321747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a358dd-9577-493d-8068-3517bba020ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:38.322059 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a358dd-9577-493d-8068-3517bba020ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:38.324284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a358dd-9577-493d-8068-3517bba020ea admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:38.327691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 991/1979] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:38 2022] DELETE /identity/v3/users/e2513cacb68643dea8991a67a52ee0ec => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:38.359837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 989/1980] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:13:38.365313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234421b-a7b8-4ad5-84d8-1b9e1b6e75a2 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:38.368475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c234421b-a7b8-4ad5-84d8-1b9e1b6e75a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:38.467135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234421b-a7b8-4ad5-84d8-1b9e1b6e75a2 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:38.473515 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c49e3-9e86-4ddb-8cc5-2d0ac4e56da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:38.474658 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c49e3-9e86-4ddb-8cc5-2d0ac4e56da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:38.475178 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c49e3-9e86-4ddb-8cc5-2d0ac4e56da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:38.475647 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c49e3-9e86-4ddb-8cc5-2d0ac4e56da1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:38.476583 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c49e3-9e86-4ddb-8cc5-2d0ac4e56da1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:38.479451 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c49e3-9e86-4ddb-8cc5-2d0ac4e56da1 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:38.507166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 992/1981] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:38 2022] POST /identity/v3/projects => generated 387 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:38.512344 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a358dd-9577-493d-8068-3517bba020ea admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:38.524074 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a358dd-9577-493d-8068-3517bba020ea admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:38.528743 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:38.596290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4dd20-04ed-4ba3-b9fd-b548e383e37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:38.596955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4dd20-04ed-4ba3-b9fd-b548e383e37d admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:38.597242 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4dd20-04ed-4ba3-b9fd-b548e383e37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:38.597661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4dd20-04ed-4ba3-b9fd-b548e383e37d admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:38.598483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf4dd20-04ed-4ba3-b9fd-b548e383e37d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:38.600479 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf4dd20-04ed-4ba3-b9fd-b548e383e37d admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:38.622392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 990/1982] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:38 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:38.626481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c49e3-9e86-4ddb-8cc5-2d0ac4e56da1 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:38.657207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c49e3-9e86-4ddb-8cc5-2d0ac4e56da1 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:38.673039 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3ccca-355f-4a8d-899b-72a31f51fc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:38.673895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3ccca-355f-4a8d-899b-72a31f51fc6e admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:38.674256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3ccca-355f-4a8d-899b-72a31f51fc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:38.674689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a3ccca-355f-4a8d-899b-72a31f51fc6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:38.675689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3ccca-355f-4a8d-899b-72a31f51fc6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:38.677634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a3ccca-355f-4a8d-899b-72a31f51fc6e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:38.695513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 993/1983] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:38.701428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4dd20-04ed-4ba3-b9fd-b548e383e37d admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:38.719485 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4dd20-04ed-4ba3-b9fd-b548e383e37d admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:38.731516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd5dc2-bed3-4520-83f3-e5e346fe010a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:38.734053 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd5dc2-bed3-4520-83f3-e5e346fe010a admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:38.734930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd5dc2-bed3-4520-83f3-e5e346fe010a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:38.737441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd5dc2-bed3-4520-83f3-e5e346fe010a admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:38.740746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd5dc2-bed3-4520-83f3-e5e346fe010a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:38.744838 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd5dc2-bed3-4520-83f3-e5e346fe010a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:38.757361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3ccca-355f-4a8d-899b-72a31f51fc6e admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:38.760085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-48a3ccca-355f-4a8d-899b-72a31f51fc6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:38.760718 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 991/1984] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:38.764787 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd5dc2-bed3-4520-83f3-e5e346fe010a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:38.776769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd5dc2-bed3-4520-83f3-e5e346fe010a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:38.782571 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56b875-7775-48a0-a1d2-af6034082875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:38.783510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56b875-7775-48a0-a1d2-af6034082875 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:38.783795 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56b875-7775-48a0-a1d2-af6034082875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:38.784070 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56b875-7775-48a0-a1d2-af6034082875 admin admin] PATH_INFO: `/v3/projects/694a55d412434a2b9b6cc882d111af27/users/6e35ffb8c2d64617ae2d3807c93a39a4/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:38.789882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a3ccca-355f-4a8d-899b-72a31f51fc6e admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:38.797310 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ddbea4-00d3-4afd-9e69-a67b25150373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:38.798209 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddbea4-00d3-4afd-9e69-a67b25150373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:38.798630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddbea4-00d3-4afd-9e69-a67b25150373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:38.798987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ddbea4-00d3-4afd-9e69-a67b25150373 admin admin] PATH_INFO: `/v3/projects/aa6ede4bbe244f068ae6e5d88bda7adf` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:38.800437 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ddbea4-00d3-4afd-9e69-a67b25150373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa6ede4bbe244f068ae6e5d88bda7adf)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:38.802595 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ddbea4-00d3-4afd-9e69-a67b25150373 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:38.804188 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56b875-7775-48a0-a1d2-af6034082875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=694a55d412434a2b9b6cc882d111af27, user_id=6e35ffb8c2d64617ae2d3807c93a39a4, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:38.807536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56b875-7775-48a0-a1d2-af6034082875 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:38.830733 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 992/1985] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:38 2022] PUT /identity/v3/projects/694a55d412434a2b9b6cc882d111af27/users/6e35ffb8c2d64617ae2d3807c93a39a4/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:38.830733 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56b875-7775-48a0-a1d2-af6034082875 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:38.830733 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9b56b875-7775-48a0-a1d2-af6034082875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:38.875297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-36ddbea4-00d3-4afd-9e69-a67b25150373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa6ede4bbe244f068ae6e5d88bda7adf', 'request_id': 'req-36ddbea4-00d3-4afd-9e69-a67b25150373'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.880981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-36ddbea4-00d3-4afd-9e69-a67b25150373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa6ede4bbe244f068ae6e5d88bda7adf', 'request_id': 'req-36ddbea4-00d3-4afd-9e69-a67b25150373'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.888568 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-36ddbea4-00d3-4afd-9e69-a67b25150373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa6ede4bbe244f068ae6e5d88bda7adf', 'request_id': 'req-36ddbea4-00d3-4afd-9e69-a67b25150373'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.899646 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-36ddbea4-00d3-4afd-9e69-a67b25150373 admin admin] The token cache is being invalidate because project aa6ede4bbe244f068ae6e5d88bda7adf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:38.899930 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-36ddbea4-00d3-4afd-9e69-a67b25150373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.900581 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-36ddbea4-00d3-4afd-9e69-a67b25150373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:38.902137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 994/1986] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:38 2022] DELETE /identity/v3/projects/aa6ede4bbe244f068ae6e5d88bda7adf => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:38.903671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56b875-7775-48a0-a1d2-af6034082875 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:38.906959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18b6d2-230b-4df6-9c8c-ac2f89355892 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:38.907252 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18b6d2-230b-4df6-9c8c-ac2f89355892 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:38.907641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18b6d2-230b-4df6-9c8c-ac2f89355892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:38.937133 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-ca18b6d2-230b-4df6-9c8c-ac2f89355892 None None] MFA Rules not processed for user `6e35ffb8c2d64617ae2d3807c93a39a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:38.938640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ca18b6d2-230b-4df6-9c8c-ac2f89355892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:38.966285 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:38.968194 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de1f7a-eb72-4778-8a53-0c51095e6dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:38.969151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de1f7a-eb72-4778-8a53-0c51095e6dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:38.969518 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de1f7a-eb72-4778-8a53-0c51095e6dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:38.969858 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de1f7a-eb72-4778-8a53-0c51095e6dd5 admin admin] PATH_INFO: `/v3/projects/29844ed4364f4bcea590e2db3a69768b` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:38.971165 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de1f7a-eb72-4778-8a53-0c51095e6dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29844ed4364f4bcea590e2db3a69768b)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:38.973178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de1f7a-eb72-4778-8a53-0c51095e6dd5 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:38.984195 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:39.015861 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 995/1987] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:39.017148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfb796-4ccf-45d4-95a2-c4f165f83a1e None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:39.023306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfb796-4ccf-45d4-95a2-c4f165f83a1e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:39.023306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfb796-4ccf-45d4-95a2-c4f165f83a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:39.038045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-66de1f7a-eb72-4778-8a53-0c51095e6dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29844ed4364f4bcea590e2db3a69768b', 'request_id': 'req-66de1f7a-eb72-4778-8a53-0c51095e6dd5'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:39.044911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-66de1f7a-eb72-4778-8a53-0c51095e6dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29844ed4364f4bcea590e2db3a69768b', 'request_id': 'req-66de1f7a-eb72-4778-8a53-0c51095e6dd5'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:39.047668 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-75dfb796-4ccf-45d4-95a2-c4f165f83a1e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:39.051040 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-75dfb796-4ccf-45d4-95a2-c4f165f83a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:39.057787 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-66de1f7a-eb72-4778-8a53-0c51095e6dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29844ed4364f4bcea590e2db3a69768b', 'request_id': 'req-66de1f7a-eb72-4778-8a53-0c51095e6dd5'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:39.081456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-66de1f7a-eb72-4778-8a53-0c51095e6dd5 admin admin] The token cache is being invalidate because project 29844ed4364f4bcea590e2db3a69768b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:39.081746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-66de1f7a-eb72-4778-8a53-0c51095e6dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:39.082542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-66de1f7a-eb72-4778-8a53-0c51095e6dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:39.085560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 993/1988] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:38 2022] DELETE /identity/v3/projects/29844ed4364f4bcea590e2db3a69768b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:39.086182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de1f7a-eb72-4778-8a53-0c51095e6dd5 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:39.088597 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-66de1f7a-eb72-4778-8a53-0c51095e6dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:39.141931 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 996/1989] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:39.142863 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfb796-4ccf-45d4-95a2-c4f165f83a1e None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:39.144996 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-75dfb796-4ccf-45d4-95a2-c4f165f83a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:39.163164 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfb796-4ccf-45d4-95a2-c4f165f83a1e None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:39.188770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d660cfd-2d1c-4058-ac59-6028b3f42581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:39.190065 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d660cfd-2d1c-4058-ac59-6028b3f42581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:39.190388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d660cfd-2d1c-4058-ac59-6028b3f42581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:39.190813 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d660cfd-2d1c-4058-ac59-6028b3f42581 admin admin] PATH_INFO: `/v3/users/69fabc59dc54451e82c52aaf155b4103` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:39.192360 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d660cfd-2d1c-4058-ac59-6028b3f42581 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69fabc59dc54451e82c52aaf155b4103)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:39.194495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d660cfd-2d1c-4058-ac59-6028b3f42581 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:39.254797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de1f7a-eb72-4778-8a53-0c51095e6dd5 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:39.266959 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe29ed8-0a01-4a5c-bd3a-ea4129a4e696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:39.267251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-3d660cfd-2d1c-4058-ac59-6028b3f42581 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69fabc59dc54451e82c52aaf155b4103'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:39.268326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe29ed8-0a01-4a5c-bd3a-ea4129a4e696 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:39.268656 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe29ed8-0a01-4a5c-bd3a-ea4129a4e696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:39.268958 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe29ed8-0a01-4a5c-bd3a-ea4129a4e696 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:39.272759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe29ed8-0a01-4a5c-bd3a-ea4129a4e696 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:39.275400 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe29ed8-0a01-4a5c-bd3a-ea4129a4e696 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:39.284626 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-3d660cfd-2d1c-4058-ac59-6028b3f42581 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69fabc59dc54451e82c52aaf155b4103'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:39.295381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-3d660cfd-2d1c-4058-ac59-6028b3f42581 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69fabc59dc54451e82c52aaf155b4103'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:39.296845 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-3d660cfd-2d1c-4058-ac59-6028b3f42581 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69fabc59dc54451e82c52aaf155b4103'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:39.307404 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-3d660cfd-2d1c-4058-ac59-6028b3f42581 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69fabc59dc54451e82c52aaf155b4103'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:39.315173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 997/1990] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:39 2022] DELETE /identity/v3/users/69fabc59dc54451e82c52aaf155b4103 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:39.316454 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05116e5b-1011-46d6-8a35-f7dae3cae845 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:39.316743 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05116e5b-1011-46d6-8a35-f7dae3cae845 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:39.317019 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05116e5b-1011-46d6-8a35-f7dae3cae845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:39.327827 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 994/1991] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:39 2022] POST /identity/v3/projects => generated 387 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:39.332767 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe29ed8-0a01-4a5c-bd3a-ea4129a4e696 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:39.337640 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5fe29ed8-0a01-4a5c-bd3a-ea4129a4e696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:39.339793 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-05116e5b-1011-46d6-8a35-f7dae3cae845 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:39.341972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-05116e5b-1011-46d6-8a35-f7dae3cae845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:39.395654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:39.447954 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 998/1992] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:39.447954 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05116e5b-1011-46d6-8a35-f7dae3cae845 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:39.462534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05116e5b-1011-46d6-8a35-f7dae3cae845 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:39.470016 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1f849-3578-4840-8b98-9e0a750da539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:39.471229 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1f849-3578-4840-8b98-9e0a750da539 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:39.472659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1f849-3578-4840-8b98-9e0a750da539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:39.472998 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1f849-3578-4840-8b98-9e0a750da539 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:39.473846 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1f849-3578-4840-8b98-9e0a750da539 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:39.474187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1f849-3578-4840-8b98-9e0a750da539 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:39.475767 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe29ed8-0a01-4a5c-bd3a-ea4129a4e696 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:39.475987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1f849-3578-4840-8b98-9e0a750da539 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:39.481616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb97f6-f6d2-4b17-b85e-f13456fd4180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:39.482537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb97f6-f6d2-4b17-b85e-f13456fd4180 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:39.482853 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb97f6-f6d2-4b17-b85e-f13456fd4180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:39.483183 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb97f6-f6d2-4b17-b85e-f13456fd4180 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:39.483891 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb97f6-f6d2-4b17-b85e-f13456fd4180 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:39.486461 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb97f6-f6d2-4b17-b85e-f13456fd4180 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:39.495109 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 999/1993] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:13:39.496077 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1f849-3578-4840-8b98-9e0a750da539 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:39.510245 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1f849-3578-4840-8b98-9e0a750da539 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:39.516333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad568853-f5f7-42c7-87a4-ec614c08d289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:39.516574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 995/1994] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:39 2022] POST /identity/v3/users => generated 300 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:39.517392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad568853-f5f7-42c7-87a4-ec614c08d289 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:39.517683 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad568853-f5f7-42c7-87a4-ec614c08d289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:39.517995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad568853-f5f7-42c7-87a4-ec614c08d289 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:39.518706 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad568853-f5f7-42c7-87a4-ec614c08d289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:39.520711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb97f6-f6d2-4b17-b85e-f13456fd4180 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:39.520921 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad568853-f5f7-42c7-87a4-ec614c08d289 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:39.543171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb97f6-f6d2-4b17-b85e-f13456fd4180 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:39.550818 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac574c0-0a83-43d0-9ba3-19198c55d6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:39.551506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac574c0-0a83-43d0-9ba3-19198c55d6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:39.551788 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac574c0-0a83-43d0-9ba3-19198c55d6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:39.552064 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac574c0-0a83-43d0-9ba3-19198c55d6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:39.552802 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac574c0-0a83-43d0-9ba3-19198c55d6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:39.554396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac574c0-0a83-43d0-9ba3-19198c55d6f8 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:39.557871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1000/1995] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:39 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:39.562323 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad568853-f5f7-42c7-87a4-ec614c08d289 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:39.574712 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 996/1996] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:39.578729 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac574c0-0a83-43d0-9ba3-19198c55d6f8 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:39.584919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:39.592459 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac574c0-0a83-43d0-9ba3-19198c55d6f8 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:39.672985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad568853-f5f7-42c7-87a4-ec614c08d289 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:39.675958 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2f79a-4924-4f11-bc94-a7b29e93d99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:39.677040 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2f79a-4924-4f11-bc94-a7b29e93d99c admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:39.677328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2f79a-4924-4f11-bc94-a7b29e93d99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:39.677714 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c2f79a-4924-4f11-bc94-a7b29e93d99c admin admin] PATH_INFO: `/v3/projects/07ffe38046f247bb8382ddbe7acbbfce/users/d9beb76ab1a24f28a640b1aec14f0791/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:39.683281 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79227ef9-cd30-42d8-b49e-4a83e8a2490e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:39.684257 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79227ef9-cd30-42d8-b49e-4a83e8a2490e admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:39.684752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79227ef9-cd30-42d8-b49e-4a83e8a2490e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:39.685384 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79227ef9-cd30-42d8-b49e-4a83e8a2490e admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:39.686365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79227ef9-cd30-42d8-b49e-4a83e8a2490e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:39.690143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79227ef9-cd30-42d8-b49e-4a83e8a2490e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:39.695993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2f79a-4924-4f11-bc94-a7b29e93d99c admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ffe38046f247bb8382ddbe7acbbfce, user_id=d9beb76ab1a24f28a640b1aec14f0791, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:39.699342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c2f79a-4924-4f11-bc94-a7b29e93d99c admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:39.725694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1001/1997] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:39 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:39.726548 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79227ef9-cd30-42d8-b49e-4a83e8a2490e admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:39.738717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-79227ef9-cd30-42d8-b49e-4a83e8a2490e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:39.738930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 997/1998] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:39 2022] PUT /identity/v3/projects/07ffe38046f247bb8382ddbe7acbbfce/users/d9beb76ab1a24f28a640b1aec14f0791/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:39.738930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2f79a-4924-4f11-bc94-a7b29e93d99c admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:39.749933 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79227ef9-cd30-42d8-b49e-4a83e8a2490e admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:39.832512 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c2f79a-4924-4f11-bc94-a7b29e93d99c admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:39.843981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c28097-bc2e-481e-9dfb-f4eec79d9a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:39.845760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c28097-bc2e-481e-9dfb-f4eec79d9a81 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:39.846046 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c28097-bc2e-481e-9dfb-f4eec79d9a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:39.851483 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e763b9-7305-4da2-adbd-1f6fd2532daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:39.852738 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c28097-bc2e-481e-9dfb-f4eec79d9a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:39.853469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e763b9-7305-4da2-adbd-1f6fd2532daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:39.853619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c28097-bc2e-481e-9dfb-f4eec79d9a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:39.853753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e763b9-7305-4da2-adbd-1f6fd2532daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:39.853834 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e763b9-7305-4da2-adbd-1f6fd2532daa admin admin] PATH_INFO: `/v3/users/06a2c82e1dbf4b508732fd83e74269e7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:39.855222 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c28097-bc2e-481e-9dfb-f4eec79d9a81 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:39.856511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e763b9-7305-4da2-adbd-1f6fd2532daa admin admin] RBAC: Authorizing `identity:delete_user(user_id=06a2c82e1dbf4b508732fd83e74269e7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:39.859633 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e763b9-7305-4da2-adbd-1f6fd2532daa admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:39.867121 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 998/1999] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:39.869890 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c28097-bc2e-481e-9dfb-f4eec79d9a81 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:39.882167 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c28097-bc2e-481e-9dfb-f4eec79d9a81 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:39.887707 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04210250-db9a-45b3-9a8d-c75ecc7be12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:39.888416 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04210250-db9a-45b3-9a8d-c75ecc7be12b admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:39.888702 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04210250-db9a-45b3-9a8d-c75ecc7be12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:39.888983 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04210250-db9a-45b3-9a8d-c75ecc7be12b admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:39.889714 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04210250-db9a-45b3-9a8d-c75ecc7be12b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:39.891361 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04210250-db9a-45b3-9a8d-c75ecc7be12b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:39.906068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 999/2000] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:39.906068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04210250-db9a-45b3-9a8d-c75ecc7be12b admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:39.907425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-04210250-db9a-45b3-9a8d-c75ecc7be12b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:39.916308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:39.925156 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04210250-db9a-45b3-9a8d-c75ecc7be12b admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:39.927227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-e6e763b9-7305-4da2-adbd-1f6fd2532daa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06a2c82e1dbf4b508732fd83e74269e7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:39.932220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597b2aa-195c-4d6d-a723-e47cb4223f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:39.934602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597b2aa-195c-4d6d-a723-e47cb4223f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:39.934905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597b2aa-195c-4d6d-a723-e47cb4223f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:39.935185 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597b2aa-195c-4d6d-a723-e47cb4223f5c admin admin] PATH_INFO: `/v3/projects/96bc418392b144d48428d5a5d00c4059` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:39.936679 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597b2aa-195c-4d6d-a723-e47cb4223f5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=96bc418392b144d48428d5a5d00c4059)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:39.938562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597b2aa-195c-4d6d-a723-e47cb4223f5c admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:39.955249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-e6e763b9-7305-4da2-adbd-1f6fd2532daa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06a2c82e1dbf4b508732fd83e74269e7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:39.964298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-e6e763b9-7305-4da2-adbd-1f6fd2532daa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06a2c82e1dbf4b508732fd83e74269e7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:39.965107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-e6e763b9-7305-4da2-adbd-1f6fd2532daa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06a2c82e1dbf4b508732fd83e74269e7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:39.973294 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-e6e763b9-7305-4da2-adbd-1f6fd2532daa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06a2c82e1dbf4b508732fd83e74269e7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:39.981672 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1002/2001] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:39 2022] DELETE /identity/v3/users/06a2c82e1dbf4b508732fd83e74269e7 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:39.982504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e763b9-7305-4da2-adbd-1f6fd2532daa admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:39.983900 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e6e763b9-7305-4da2-adbd-1f6fd2532daa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:39.996923 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c597b2aa-195c-4d6d-a723-e47cb4223f5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96bc418392b144d48428d5a5d00c4059', 'request_id': 'req-c597b2aa-195c-4d6d-a723-e47cb4223f5c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:40.002835 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c597b2aa-195c-4d6d-a723-e47cb4223f5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96bc418392b144d48428d5a5d00c4059', 'request_id': 'req-c597b2aa-195c-4d6d-a723-e47cb4223f5c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:40.010165 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c597b2aa-195c-4d6d-a723-e47cb4223f5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96bc418392b144d48428d5a5d00c4059', 'request_id': 'req-c597b2aa-195c-4d6d-a723-e47cb4223f5c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:40.013063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:40.025674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c597b2aa-195c-4d6d-a723-e47cb4223f5c admin admin] The token cache is being invalidate because project 96bc418392b144d48428d5a5d00c4059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:40.027367 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c597b2aa-195c-4d6d-a723-e47cb4223f5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:40.028312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c597b2aa-195c-4d6d-a723-e47cb4223f5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:40.033232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1000/2002] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:39 2022] DELETE /identity/v3/projects/96bc418392b144d48428d5a5d00c4059 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:40.033232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597b2aa-195c-4d6d-a723-e47cb4223f5c admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:40.033232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c597b2aa-195c-4d6d-a723-e47cb4223f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:40.071279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e763b9-7305-4da2-adbd-1f6fd2532daa admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:40.076911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b6678-80a9-47ba-ad51-fa1d366adb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:40.077717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b6678-80a9-47ba-ad51-fa1d366adb91 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:40.078013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b6678-80a9-47ba-ad51-fa1d366adb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:40.078289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b6678-80a9-47ba-ad51-fa1d366adb91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:40.079239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b6678-80a9-47ba-ad51-fa1d366adb91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:40.080848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b6678-80a9-47ba-ad51-fa1d366adb91 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:40.096500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1003/2003] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:40.096500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1a0a5-0b6b-4491-9742-f22934992016 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:40.096500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1a0a5-0b6b-4491-9742-f22934992016 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:40.096500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1a0a5-0b6b-4491-9742-f22934992016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:40.110752 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597b2aa-195c-4d6d-a723-e47cb4223f5c admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:40.116093 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b75d51-f952-4d79-a982-8ef4c42c3691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:40.120143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-ace1a0a5-0b6b-4491-9742-f22934992016 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:40.120304 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b75d51-f952-4d79-a982-8ef4c42c3691 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:40.120695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b75d51-f952-4d79-a982-8ef4c42c3691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:40.121192 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b75d51-f952-4d79-a982-8ef4c42c3691 admin admin] PATH_INFO: `/v3/projects/07ffe38046f247bb8382ddbe7acbbfce/users/d9beb76ab1a24f28a640b1aec14f0791/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:40.121495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ace1a0a5-0b6b-4491-9742-f22934992016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:40.123655 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b75d51-f952-4d79-a982-8ef4c42c3691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ffe38046f247bb8382ddbe7acbbfce, user_id=d9beb76ab1a24f28a640b1aec14f0791, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:40.126303 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b75d51-f952-4d79-a982-8ef4c42c3691 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:40.132434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1004/2004] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:40.134548 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9f6d3-3b92-48f9-9a98-abd1deae3495 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:40.134950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9f6d3-3b92-48f9-9a98-abd1deae3495 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:40.135232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9f6d3-3b92-48f9-9a98-abd1deae3495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:40.153617 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1001/2005] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:40 2022] PUT /identity/v3/projects/07ffe38046f247bb8382ddbe7acbbfce/users/d9beb76ab1a24f28a640b1aec14f0791/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:40.154947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b75d51-f952-4d79-a982-8ef4c42c3691 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:40.157554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-fca9f6d3-3b92-48f9-9a98-abd1deae3495 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:40.157948 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-81b75d51-f952-4d79-a982-8ef4c42c3691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:40.158882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-fca9f6d3-3b92-48f9-9a98-abd1deae3495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:40.252571 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1005/2006] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:40.253259 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9f6d3-3b92-48f9-9a98-abd1deae3495 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:40.254729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-fca9f6d3-3b92-48f9-9a98-abd1deae3495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:40.260919 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b75d51-f952-4d79-a982-8ef4c42c3691 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:40.265600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedc8db-c0ab-4782-bcc0-373799ab4d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:40.266534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedc8db-c0ab-4782-bcc0-373799ab4d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:40.266934 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedc8db-c0ab-4782-bcc0-373799ab4d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:40.267203 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedc8db-c0ab-4782-bcc0-373799ab4d6b admin admin] PATH_INFO: `/v3/projects/346ff0b059fa4bc2b4b8426ff8ba71bf/users/ad43df748f09486181a0cc764fb004b1/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:40.285282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedc8db-c0ab-4782-bcc0-373799ab4d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=346ff0b059fa4bc2b4b8426ff8ba71bf, user_id=ad43df748f09486181a0cc764fb004b1, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:40.288392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedc8db-c0ab-4782-bcc0-373799ab4d6b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:40.307335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1002/2007] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:40 2022] PUT /identity/v3/projects/346ff0b059fa4bc2b4b8426ff8ba71bf/users/ad43df748f09486181a0cc764fb004b1/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:40.308761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e86ef6-4463-4b6d-a61c-01f3c2ed4a87 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:40.309062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e86ef6-4463-4b6d-a61c-01f3c2ed4a87 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:40.309442 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e86ef6-4463-4b6d-a61c-01f3c2ed4a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:40.315546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:40.333709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9f6d3-3b92-48f9-9a98-abd1deae3495 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:40.342644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-24e86ef6-4463-4b6d-a61c-01f3c2ed4a87 None None] MFA Rules not processed for user `d9beb76ab1a24f28a640b1aec14f0791`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:40.343932 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-24e86ef6-4463-4b6d-a61c-01f3c2ed4a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:40.395517 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08a76d-0283-4822-96d7-5b4edb848a2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:40.396702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08a76d-0283-4822-96d7-5b4edb848a2f service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:40.396996 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08a76d-0283-4822-96d7-5b4edb848a2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:40.397273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08a76d-0283-4822-96d7-5b4edb848a2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:40.420248 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:40.434837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1003/2008] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:40.436599 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5236d9-1048-4bee-9628-7ae7b2f7c9ca None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:40.437053 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5236d9-1048-4bee-9628-7ae7b2f7c9ca None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:40.437644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5236d9-1048-4bee-9628-7ae7b2f7c9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:40.463165 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-de5236d9-1048-4bee-9628-7ae7b2f7c9ca None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:40.466210 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-de5236d9-1048-4bee-9628-7ae7b2f7c9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:40.491757 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08a76d-0283-4822-96d7-5b4edb848a2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:40.493363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08a76d-0283-4822-96d7-5b4edb848a2f service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:40.504490 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1006/2009] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:40.505137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423ee9a-fa62-41a9-9684-49a0f042e1ff None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:40.505427 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423ee9a-fa62-41a9-9684-49a0f042e1ff None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:40.505705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423ee9a-fa62-41a9-9684-49a0f042e1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:40.546120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-6423ee9a-fa62-41a9-9684-49a0f042e1ff None None] MFA Rules not processed for user `ad43df748f09486181a0cc764fb004b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:40.547840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6423ee9a-fa62-41a9-9684-49a0f042e1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:40.583567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1004/2010] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:40.585062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5236d9-1048-4bee-9628-7ae7b2f7c9ca None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:40.588312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-de5236d9-1048-4bee-9628-7ae7b2f7c9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:40.643083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1007/2011] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:40.644283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423ee9a-fa62-41a9-9684-49a0f042e1ff None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:40.665613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423ee9a-fa62-41a9-9684-49a0f042e1ff None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:40.672037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea4bd4-d24a-483d-8df7-758d592b32ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:40.672723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea4bd4-d24a-483d-8df7-758d592b32ff admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:40.673021 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea4bd4-d24a-483d-8df7-758d592b32ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:40.673334 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea4bd4-d24a-483d-8df7-758d592b32ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:40.674081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea4bd4-d24a-483d-8df7-758d592b32ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:40.674438 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea4bd4-d24a-483d-8df7-758d592b32ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:40.676147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea4bd4-d24a-483d-8df7-758d592b32ff admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:40.690264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5236d9-1048-4bee-9628-7ae7b2f7c9ca None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:40.694913 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae29b4-feb3-4a64-a45e-b7282b9e9a69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:40.696284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae29b4-feb3-4a64-a45e-b7282b9e9a69 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:40.696699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae29b4-feb3-4a64-a45e-b7282b9e9a69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:40.697058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae29b4-feb3-4a64-a45e-b7282b9e9a69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:40.697175 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1008/2012] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:13:40.699254 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea4bd4-d24a-483d-8df7-758d592b32ff admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:40.710467 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea4bd4-d24a-483d-8df7-758d592b32ff admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:40.712881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae29b4-feb3-4a64-a45e-b7282b9e9a69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:40.715476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cae29b4-feb3-4a64-a45e-b7282b9e9a69 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:40.717125 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b18a1b-bd42-492d-818a-4d665aaccfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:40.718010 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b18a1b-bd42-492d-818a-4d665aaccfa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:40.718623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b18a1b-bd42-492d-818a-4d665aaccfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:40.719420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b18a1b-bd42-492d-818a-4d665aaccfa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:40.720137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b18a1b-bd42-492d-818a-4d665aaccfa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:40.722317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b18a1b-bd42-492d-818a-4d665aaccfa5 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:40.731066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1005/2013] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:40.731908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae29b4-feb3-4a64-a45e-b7282b9e9a69 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:40.744676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cae29b4-feb3-4a64-a45e-b7282b9e9a69 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:40.751465 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f92557-ec42-4c96-a07c-7ae7da8c2ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:40.752146 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f92557-ec42-4c96-a07c-7ae7da8c2ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:40.752430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f92557-ec42-4c96-a07c-7ae7da8c2ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:40.752710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f92557-ec42-4c96-a07c-7ae7da8c2ec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:40.753552 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f92557-ec42-4c96-a07c-7ae7da8c2ec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:40.755965 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f92557-ec42-4c96-a07c-7ae7da8c2ec3 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:40.778274 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1009/2014] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:40 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:40.779594 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b18a1b-bd42-492d-818a-4d665aaccfa5 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:40.787492 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:40.800224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1006/2015] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:40 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:40.801053 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f92557-ec42-4c96-a07c-7ae7da8c2ec3 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:40.923071 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f92557-ec42-4c96-a07c-7ae7da8c2ec3 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:40.933022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e87c7d-a070-46fb-8b54-8722c2567ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:40.934069 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e87c7d-a070-46fb-8b54-8722c2567ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:40.934501 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e87c7d-a070-46fb-8b54-8722c2567ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:40.934961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e87c7d-a070-46fb-8b54-8722c2567ed8 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:40.937623 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e87c7d-a070-46fb-8b54-8722c2567ed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:40.940560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e87c7d-a070-46fb-8b54-8722c2567ed8 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:40.941005 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b18a1b-bd42-492d-818a-4d665aaccfa5 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:40.969656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccf607-2634-44f2-9990-754f9d710a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:40.972542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccf607-2634-44f2-9990-754f9d710a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:40.972852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccf607-2634-44f2-9990-754f9d710a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:40.974518 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccf607-2634-44f2-9990-754f9d710a5f admin admin] PATH_INFO: `/v3/users/6e35ffb8c2d64617ae2d3807c93a39a4` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:40.979268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ccf607-2634-44f2-9990-754f9d710a5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e35ffb8c2d64617ae2d3807c93a39a4)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:40.986564 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ccf607-2634-44f2-9990-754f9d710a5f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:40.997167 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1007/2016] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:40 2022] POST /identity/v3/users => generated 297 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:40.998945 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e87c7d-a070-46fb-8b54-8722c2567ed8 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:41.033782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e87c7d-a070-46fb-8b54-8722c2567ed8 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:41.051427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be1a62-2b61-4e41-af8e-7969e2504842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:41.053459 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be1a62-2b61-4e41-af8e-7969e2504842 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:41.053821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be1a62-2b61-4e41-af8e-7969e2504842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:41.056779 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be1a62-2b61-4e41-af8e-7969e2504842 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:41.058434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be1a62-2b61-4e41-af8e-7969e2504842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:41.064746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be1a62-2b61-4e41-af8e-7969e2504842 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:41.081386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-e9ccf607-2634-44f2-9990-754f9d710a5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e35ffb8c2d64617ae2d3807c93a39a4'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:41.082287 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:41.098199 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-e9ccf607-2634-44f2-9990-754f9d710a5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e35ffb8c2d64617ae2d3807c93a39a4'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:41.108630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-e9ccf607-2634-44f2-9990-754f9d710a5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e35ffb8c2d64617ae2d3807c93a39a4'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:41.109604 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-e9ccf607-2634-44f2-9990-754f9d710a5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e35ffb8c2d64617ae2d3807c93a39a4'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:41.113276 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1008/2017] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:40 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:41.114378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be1a62-2b61-4e41-af8e-7969e2504842 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:41.119635 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-e9ccf607-2634-44f2-9990-754f9d710a5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e35ffb8c2d64617ae2d3807c93a39a4'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:41.131647 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1010/2018] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:40 2022] DELETE /identity/v3/users/6e35ffb8c2d64617ae2d3807c93a39a4 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:41.132544 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccf607-2634-44f2-9990-754f9d710a5f admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:41.134080 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e9ccf607-2634-44f2-9990-754f9d710a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:41.150902 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be1a62-2b61-4e41-af8e-7969e2504842 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:41.237047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccf607-2634-44f2-9990-754f9d710a5f admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:41.239530 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57722b0-682b-4619-bacb-b477b6991a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:41.240544 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57722b0-682b-4619-bacb-b477b6991a3e admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:41.241002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57722b0-682b-4619-bacb-b477b6991a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:41.241405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57722b0-682b-4619-bacb-b477b6991a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:41.242530 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57722b0-682b-4619-bacb-b477b6991a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:41.244057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd250089-378f-4c55-aff3-0ba936c5f452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:41.245147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd250089-378f-4c55-aff3-0ba936c5f452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:41.245435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd250089-378f-4c55-aff3-0ba936c5f452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:41.245742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd250089-378f-4c55-aff3-0ba936c5f452 admin admin] PATH_INFO: `/v3/projects/d6de81c54cc74648a90058199ee3f899` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:41.246805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57722b0-682b-4619-bacb-b477b6991a3e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:41.247029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd250089-378f-4c55-aff3-0ba936c5f452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6de81c54cc74648a90058199ee3f899)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:41.249141 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd250089-378f-4c55-aff3-0ba936c5f452 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:41.266169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1009/2019] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:41.266169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57722b0-682b-4619-bacb-b477b6991a3e admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:41.266169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f57722b0-682b-4619-bacb-b477b6991a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:41.284305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57722b0-682b-4619-bacb-b477b6991a3e admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:41.293805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53540e8-dba3-4124-b0ad-0cbb72b854e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:41.294472 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:41.296403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53540e8-dba3-4124-b0ad-0cbb72b854e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:41.296975 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53540e8-dba3-4124-b0ad-0cbb72b854e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:41.297426 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53540e8-dba3-4124-b0ad-0cbb72b854e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:41.298494 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53540e8-dba3-4124-b0ad-0cbb72b854e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:41.302453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53540e8-dba3-4124-b0ad-0cbb72b854e7 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:41.316878 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1010/2020] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:41.318360 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422890ad-d96f-4ee0-b540-6a42dd96caeb None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:41.318653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422890ad-d96f-4ee0-b540-6a42dd96caeb None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:41.318963 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422890ad-d96f-4ee0-b540-6a42dd96caeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:41.322819 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cd250089-378f-4c55-aff3-0ba936c5f452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6de81c54cc74648a90058199ee3f899', 'request_id': 'req-cd250089-378f-4c55-aff3-0ba936c5f452'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:41.331485 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cd250089-378f-4c55-aff3-0ba936c5f452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6de81c54cc74648a90058199ee3f899', 'request_id': 'req-cd250089-378f-4c55-aff3-0ba936c5f452'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:41.343974 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cd250089-378f-4c55-aff3-0ba936c5f452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6de81c54cc74648a90058199ee3f899', 'request_id': 'req-cd250089-378f-4c55-aff3-0ba936c5f452'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:41.356871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-422890ad-d96f-4ee0-b540-6a42dd96caeb None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:41.359725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-422890ad-d96f-4ee0-b540-6a42dd96caeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:41.364124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cd250089-378f-4c55-aff3-0ba936c5f452 admin admin] The token cache is being invalidate because project d6de81c54cc74648a90058199ee3f899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:41.364542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cd250089-378f-4c55-aff3-0ba936c5f452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:41.366044 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cd250089-378f-4c55-aff3-0ba936c5f452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:41.369618 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1011/2021] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:41 2022] DELETE /identity/v3/projects/d6de81c54cc74648a90058199ee3f899 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:41.371585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd250089-378f-4c55-aff3-0ba936c5f452 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:41.375415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-cd250089-378f-4c55-aff3-0ba936c5f452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:41.455717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1011/2022] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:41.457606 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422890ad-d96f-4ee0-b540-6a42dd96caeb None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:41.459209 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-422890ad-d96f-4ee0-b540-6a42dd96caeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:41.474515 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422890ad-d96f-4ee0-b540-6a42dd96caeb None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:41.480567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17108767-6bef-4b66-b345-70a8526dedba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:41.482628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17108767-6bef-4b66-b345-70a8526dedba admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:41.482961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17108767-6bef-4b66-b345-70a8526dedba admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:41.483331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17108767-6bef-4b66-b345-70a8526dedba admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:41.484449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17108767-6bef-4b66-b345-70a8526dedba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:41.486489 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17108767-6bef-4b66-b345-70a8526dedba admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:41.499074 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1012/2023] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:41.500025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17108767-6bef-4b66-b345-70a8526dedba admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:41.502155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-17108767-6bef-4b66-b345-70a8526dedba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:41.513112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd250089-378f-4c55-aff3-0ba936c5f452 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:41.516456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17108767-6bef-4b66-b345-70a8526dedba admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:41.521596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168acf6d-75a5-4f18-9b12-1e4f8ae260d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:41.522491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168acf6d-75a5-4f18-9b12-1e4f8ae260d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:41.522815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168acf6d-75a5-4f18-9b12-1e4f8ae260d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:41.523097 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168acf6d-75a5-4f18-9b12-1e4f8ae260d8 admin admin] PATH_INFO: `/v3/users/3c4c31aa59b54ecda17f76b20cbd1e64` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:41.525219 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168acf6d-75a5-4f18-9b12-1e4f8ae260d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c4c31aa59b54ecda17f76b20cbd1e64)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:41.526771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f166e5b-56ad-4a9f-9f24-df0a54d4f7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:41.527261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168acf6d-75a5-4f18-9b12-1e4f8ae260d8 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:41.528131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f166e5b-56ad-4a9f-9f24-df0a54d4f7d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:41.528574 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f166e5b-56ad-4a9f-9f24-df0a54d4f7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:41.529068 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f166e5b-56ad-4a9f-9f24-df0a54d4f7d6 admin admin] PATH_INFO: `/v3/projects/bed878d897ea4974b46535b0d11cf26b/users/f021ca5d232f4c3e9ef9ee6ed86b5744/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:41.559540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f166e5b-56ad-4a9f-9f24-df0a54d4f7d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed878d897ea4974b46535b0d11cf26b, user_id=f021ca5d232f4c3e9ef9ee6ed86b5744, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:41.563355 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f166e5b-56ad-4a9f-9f24-df0a54d4f7d6 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:41.585837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1012/2024] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:41 2022] PUT /identity/v3/projects/bed878d897ea4974b46535b0d11cf26b/users/f021ca5d232f4c3e9ef9ee6ed86b5744/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:41.588257 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f166e5b-56ad-4a9f-9f24-df0a54d4f7d6 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:41.618542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-168acf6d-75a5-4f18-9b12-1e4f8ae260d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c4c31aa59b54ecda17f76b20cbd1e64'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:41.636346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:41.644724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-168acf6d-75a5-4f18-9b12-1e4f8ae260d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c4c31aa59b54ecda17f76b20cbd1e64'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:41.655010 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-168acf6d-75a5-4f18-9b12-1e4f8ae260d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c4c31aa59b54ecda17f76b20cbd1e64'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:41.655778 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-168acf6d-75a5-4f18-9b12-1e4f8ae260d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c4c31aa59b54ecda17f76b20cbd1e64'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:41.665990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-168acf6d-75a5-4f18-9b12-1e4f8ae260d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c4c31aa59b54ecda17f76b20cbd1e64'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:41.689446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1013/2025] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:41 2022] DELETE /identity/v3/users/3c4c31aa59b54ecda17f76b20cbd1e64 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:41.690375 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168acf6d-75a5-4f18-9b12-1e4f8ae260d8 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:41.694455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-168acf6d-75a5-4f18-9b12-1e4f8ae260d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:41.715188 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f166e5b-56ad-4a9f-9f24-df0a54d4f7d6 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:41.788622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168acf6d-75a5-4f18-9b12-1e4f8ae260d8 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:41.795216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b5717-af27-4ff8-90f9-40dbfbeb9c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:41.796660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b5717-af27-4ff8-90f9-40dbfbeb9c3b service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:41.796660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b5717-af27-4ff8-90f9-40dbfbeb9c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:41.797917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b5717-af27-4ff8-90f9-40dbfbeb9c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:41.805291 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8c5b5717-af27-4ff8-90f9-40dbfbeb9c3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:41.811337 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:41.840506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c4817-d3bb-4281-afa1-3244815b6889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:41.841610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c4817-d3bb-4281-afa1-3244815b6889 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:41.841905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c4817-d3bb-4281-afa1-3244815b6889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:41.842388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c4817-d3bb-4281-afa1-3244815b6889 admin admin] PATH_INFO: `/v3/projects/4d2996f0d19a4f0685bc1426b9e5f280/users/27a37a3257d04667a75f86e718235c1f/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:41.864574 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c4817-d3bb-4281-afa1-3244815b6889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d2996f0d19a4f0685bc1426b9e5f280, user_id=27a37a3257d04667a75f86e718235c1f, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:41.868248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c4817-d3bb-4281-afa1-3244815b6889 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:41.889659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1013/2026] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:41 2022] PUT /identity/v3/projects/4d2996f0d19a4f0685bc1426b9e5f280/users/27a37a3257d04667a75f86e718235c1f/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:41.890431 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c4817-d3bb-4281-afa1-3244815b6889 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:41.892112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d27c4817-d3bb-4281-afa1-3244815b6889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:41.920697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b5717-af27-4ff8-90f9-40dbfbeb9c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:41.923049 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b5717-af27-4ff8-90f9-40dbfbeb9c3b service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:41.947743 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1014/2027] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:41.949418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e2a9-15cb-4859-b632-d4513d23c1d3 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:41.949786 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e2a9-15cb-4859-b632-d4513d23c1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:41.950389 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e2a9-15cb-4859-b632-d4513d23c1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:41.980055 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c4817-d3bb-4281-afa1-3244815b6889 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:41.987190 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851af3e-d2f6-4379-836c-9f19a0b8a98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:41.987887 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851af3e-d2f6-4379-836c-9f19a0b8a98a admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:41.988497 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851af3e-d2f6-4379-836c-9f19a0b8a98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:41.988497 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851af3e-d2f6-4379-836c-9f19a0b8a98a admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:41.989171 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851af3e-d2f6-4379-836c-9f19a0b8a98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:41.991737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851af3e-d2f6-4379-836c-9f19a0b8a98a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:41.994192 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-fe02e2a9-15cb-4859-b632-d4513d23c1d3 None None] MFA Rules not processed for user `27a37a3257d04667a75f86e718235c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:41.996037 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-fe02e2a9-15cb-4859-b632-d4513d23c1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:42.007232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1014/2028] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:42.012159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851af3e-d2f6-4379-836c-9f19a0b8a98a admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:42.023765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851af3e-d2f6-4379-836c-9f19a0b8a98a admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:42.032922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7a010-4de0-47f6-9caf-a4fa21eb42da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:42.033844 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7a010-4de0-47f6-9caf-a4fa21eb42da admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:42.034129 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7a010-4de0-47f6-9caf-a4fa21eb42da admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:42.034412 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7a010-4de0-47f6-9caf-a4fa21eb42da admin admin] PATH_INFO: `/v3/projects/bed878d897ea4974b46535b0d11cf26b/users/f021ca5d232f4c3e9ef9ee6ed86b5744/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:42.036487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7a010-4de0-47f6-9caf-a4fa21eb42da admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed878d897ea4974b46535b0d11cf26b, user_id=f021ca5d232f4c3e9ef9ee6ed86b5744, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:42.041567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7a010-4de0-47f6-9caf-a4fa21eb42da admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:42.059436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1015/2029] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:42 2022] PUT /identity/v3/projects/bed878d897ea4974b46535b0d11cf26b/users/f021ca5d232f4c3e9ef9ee6ed86b5744/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:42.075402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011b9cc-3188-4982-b4ec-6637a0c78f1a None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:42.075699 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011b9cc-3188-4982-b4ec-6637a0c78f1a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:42.075986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011b9cc-3188-4982-b4ec-6637a0c78f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:42.097943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1015/2030] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:42.110962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02e2a9-15cb-4859-b632-d4513d23c1d3 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:42.112466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-fe02e2a9-15cb-4859-b632-d4513d23c1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:42.118242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-1011b9cc-3188-4982-b4ec-6637a0c78f1a None None] MFA Rules not processed for user `f021ca5d232f4c3e9ef9ee6ed86b5744`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:42.119664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-1011b9cc-3188-4982-b4ec-6637a0c78f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:42.137217 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:42.225897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1016/2031] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:42.237837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011b9cc-3188-4982-b4ec-6637a0c78f1a None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:42.239518 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-1011b9cc-3188-4982-b4ec-6637a0c78f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:42.250214 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02e2a9-15cb-4859-b632-d4513d23c1d3 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:42.334377 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6506098-6a53-418a-8969-de7eabc6cd40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:42.335455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6506098-6a53-418a-8969-de7eabc6cd40 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:42.336509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6506098-6a53-418a-8969-de7eabc6cd40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:42.337234 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6506098-6a53-418a-8969-de7eabc6cd40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:42.354058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:42.399820 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011b9cc-3188-4982-b4ec-6637a0c78f1a None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:42.409713 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cbd9b-5f82-4f2d-abf2-d9babb116eb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:42.410524 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cbd9b-5f82-4f2d-abf2-d9babb116eb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:42.410977 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cbd9b-5f82-4f2d-abf2-d9babb116eb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:42.411361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817cbd9b-5f82-4f2d-abf2-d9babb116eb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:42.425064 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817cbd9b-5f82-4f2d-abf2-d9babb116eb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:42.428018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817cbd9b-5f82-4f2d-abf2-d9babb116eb1 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:42.428900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6506098-6a53-418a-8969-de7eabc6cd40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:42.432448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6506098-6a53-418a-8969-de7eabc6cd40 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:42.442815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1017/2032] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:42.458754 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1016/2033] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:42.497064 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cbd9b-5f82-4f2d-abf2-d9babb116eb1 service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:42.499167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-817cbd9b-5f82-4f2d-abf2-d9babb116eb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:42.637930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6506098-6a53-418a-8969-de7eabc6cd40 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:42.642990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817cbd9b-5f82-4f2d-abf2-d9babb116eb1 service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:42.650746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6506098-6a53-418a-8969-de7eabc6cd40 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:42.655608 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34adab83-a697-4277-89eb-8016c94e2f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:42.656702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adab83-a697-4277-89eb-8016c94e2f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:42.656987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adab83-a697-4277-89eb-8016c94e2f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:42.657267 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adab83-a697-4277-89eb-8016c94e2f23 admin admin] PATH_INFO: `/v3/users/27a37a3257d04667a75f86e718235c1f` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:42.659463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34adab83-a697-4277-89eb-8016c94e2f23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27a37a3257d04667a75f86e718235c1f)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:42.659712 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de2c1a-c232-4987-8007-6f440ffdb896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:42.660498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de2c1a-c232-4987-8007-6f440ffdb896 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:42.661498 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34adab83-a697-4277-89eb-8016c94e2f23 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:42.663460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de2c1a-c232-4987-8007-6f440ffdb896 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:42.663873 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de2c1a-c232-4987-8007-6f440ffdb896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:42.677375 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b1de2c1a-c232-4987-8007-6f440ffdb896 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:42.782357 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-34adab83-a697-4277-89eb-8016c94e2f23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27a37a3257d04667a75f86e718235c1f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:42.809945 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-34adab83-a697-4277-89eb-8016c94e2f23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27a37a3257d04667a75f86e718235c1f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:42.812758 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de2c1a-c232-4987-8007-6f440ffdb896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:42.815574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de2c1a-c232-4987-8007-6f440ffdb896 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:42.829372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-34adab83-a697-4277-89eb-8016c94e2f23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27a37a3257d04667a75f86e718235c1f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:42.830351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-34adab83-a697-4277-89eb-8016c94e2f23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27a37a3257d04667a75f86e718235c1f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:42.843376 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-34adab83-a697-4277-89eb-8016c94e2f23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27a37a3257d04667a75f86e718235c1f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:42.855431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1017/2035] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:42.857688 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1018/2035] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:42 2022] DELETE /identity/v3/users/27a37a3257d04667a75f86e718235c1f => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:42.887458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082478d-9a95-4730-9b2d-b253b430af47 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:42.888836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082478d-9a95-4730-9b2d-b253b430af47 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:42.889805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082478d-9a95-4730-9b2d-b253b430af47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:42.912978 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34adab83-a697-4277-89eb-8016c94e2f23 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:42.915101 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-34adab83-a697-4277-89eb-8016c94e2f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:43.002989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-6082478d-9a95-4730-9b2d-b253b430af47 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:43.006583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6082478d-9a95-4730-9b2d-b253b430af47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:43.045560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:43.105919 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:43.124318 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34adab83-a697-4277-89eb-8016c94e2f23 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:43.129464 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1018/2036] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:43.129952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082478d-9a95-4730-9b2d-b253b430af47 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:43.131423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6082478d-9a95-4730-9b2d-b253b430af47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:43.145126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a14c45-3b62-4705-acbb-9ca735be6903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:43.146623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a14c45-3b62-4705-acbb-9ca735be6903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:43.146936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a14c45-3b62-4705-acbb-9ca735be6903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:43.147217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a14c45-3b62-4705-acbb-9ca735be6903 admin admin] PATH_INFO: `/v3/projects/e9bd6686cee94ae5aedc413c4fc6228d` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:43.150269 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082478d-9a95-4730-9b2d-b253b430af47 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:43.153283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a14c45-3b62-4705-acbb-9ca735be6903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9bd6686cee94ae5aedc413c4fc6228d)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:43.155258 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:43.156160 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:43.156454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:43.156739 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94 admin admin] PATH_INFO: `/v3/projects/694a55d412434a2b9b6cc882d111af27` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:43.157945 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=694a55d412434a2b9b6cc882d111af27)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:43.159923 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:43.165917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a14c45-3b62-4705-acbb-9ca735be6903 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:43.233233 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '694a55d412434a2b9b6cc882d111af27', 'request_id': 'req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.246848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d1a14c45-3b62-4705-acbb-9ca735be6903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9bd6686cee94ae5aedc413c4fc6228d', 'request_id': 'req-d1a14c45-3b62-4705-acbb-9ca735be6903'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.251654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '694a55d412434a2b9b6cc882d111af27', 'request_id': 'req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.263880 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d1a14c45-3b62-4705-acbb-9ca735be6903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9bd6686cee94ae5aedc413c4fc6228d', 'request_id': 'req-d1a14c45-3b62-4705-acbb-9ca735be6903'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.268307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '694a55d412434a2b9b6cc882d111af27', 'request_id': 'req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.279589 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d1a14c45-3b62-4705-acbb-9ca735be6903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9bd6686cee94ae5aedc413c4fc6228d', 'request_id': 'req-d1a14c45-3b62-4705-acbb-9ca735be6903'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.296390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94 admin admin] The token cache is being invalidate because project 694a55d412434a2b9b6cc882d111af27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:43.296692 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.305091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.307925 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1019/2037] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:43 2022] DELETE /identity/v3/projects/694a55d412434a2b9b6cc882d111af27 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:43.308763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:43.310141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:43.316718 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d1a14c45-3b62-4705-acbb-9ca735be6903 admin admin] The token cache is being invalidate because project e9bd6686cee94ae5aedc413c4fc6228d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:43.317012 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d1a14c45-3b62-4705-acbb-9ca735be6903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.318424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d1a14c45-3b62-4705-acbb-9ca735be6903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.324911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1019/2038] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:42 2022] DELETE /identity/v3/projects/e9bd6686cee94ae5aedc413c4fc6228d => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:43.325263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a14c45-3b62-4705-acbb-9ca735be6903 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:43.330691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d1a14c45-3b62-4705-acbb-9ca735be6903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:43.481830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166cda7-dee1-49b6-b7e7-52eda8ebaf94 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:43.513698 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a14c45-3b62-4705-acbb-9ca735be6903 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:43.535028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b9952-be3c-4844-94fe-95a7968cae10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:43.536178 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b9952-be3c-4844-94fe-95a7968cae10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:43.536617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b9952-be3c-4844-94fe-95a7968cae10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:43.537166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b9952-be3c-4844-94fe-95a7968cae10 admin admin] PATH_INFO: `/v3/users/d9beb76ab1a24f28a640b1aec14f0791` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:43.540584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b9952-be3c-4844-94fe-95a7968cae10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9beb76ab1a24f28a640b1aec14f0791)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:43.542907 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b9952-be3c-4844-94fe-95a7968cae10 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:43.596652 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531b426-995a-405e-a276-d9a9a3411514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:43.597323 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531b426-995a-405e-a276-d9a9a3411514 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:43.597601 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531b426-995a-405e-a276-d9a9a3411514 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:43.597886 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531b426-995a-405e-a276-d9a9a3411514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:43.599923 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6531b426-995a-405e-a276-d9a9a3411514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:43.616228 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531b426-995a-405e-a276-d9a9a3411514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:43.617593 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f12b9952-be3c-4844-94fe-95a7968cae10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9beb76ab1a24f28a640b1aec14f0791'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.617904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531b426-995a-405e-a276-d9a9a3411514 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:43.629364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1020/2039] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:43.641367 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531b426-995a-405e-a276-d9a9a3411514 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:43.647107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6531b426-995a-405e-a276-d9a9a3411514 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:43.673341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531b426-995a-405e-a276-d9a9a3411514 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:43.683873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f12b9952-be3c-4844-94fe-95a7968cae10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9beb76ab1a24f28a640b1aec14f0791'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.685910 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c267f4-6f02-4ae8-8170-162541ac0026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:43.689621 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c267f4-6f02-4ae8-8170-162541ac0026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:43.691097 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c267f4-6f02-4ae8-8170-162541ac0026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:43.691391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c267f4-6f02-4ae8-8170-162541ac0026 admin admin] PATH_INFO: `/v3/users/828a7185e7c44699a49d976341d15f3a` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:43.694589 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c267f4-6f02-4ae8-8170-162541ac0026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=828a7185e7c44699a49d976341d15f3a)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:43.699551 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c267f4-6f02-4ae8-8170-162541ac0026 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:43.702663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:43.705348 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f12b9952-be3c-4844-94fe-95a7968cae10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9beb76ab1a24f28a640b1aec14f0791'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.707520 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f12b9952-be3c-4844-94fe-95a7968cae10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9beb76ab1a24f28a640b1aec14f0791'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.734501 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f12b9952-be3c-4844-94fe-95a7968cae10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9beb76ab1a24f28a640b1aec14f0791'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.746509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1020/2040] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:43 2022] DELETE /identity/v3/users/d9beb76ab1a24f28a640b1aec14f0791 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:43.815425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-f3c267f4-6f02-4ae8-8170-162541ac0026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '828a7185e7c44699a49d976341d15f3a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.837156 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-f3c267f4-6f02-4ae8-8170-162541ac0026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '828a7185e7c44699a49d976341d15f3a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.847006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-f3c267f4-6f02-4ae8-8170-162541ac0026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '828a7185e7c44699a49d976341d15f3a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.848505 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-f3c267f4-6f02-4ae8-8170-162541ac0026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '828a7185e7c44699a49d976341d15f3a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.862419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-f3c267f4-6f02-4ae8-8170-162541ac0026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '828a7185e7c44699a49d976341d15f3a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:43.874737 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1021/2041] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:43 2022] DELETE /identity/v3/users/828a7185e7c44699a49d976341d15f3a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:44.402275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b9952-be3c-4844-94fe-95a7968cae10 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:44.403956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f12b9952-be3c-4844-94fe-95a7968cae10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:44.423034 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:44.486356 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c267f4-6f02-4ae8-8170-162541ac0026 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:44.489005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f3c267f4-6f02-4ae8-8170-162541ac0026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:44.529411 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b9952-be3c-4844-94fe-95a7968cae10 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:44.601425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c267f4-6f02-4ae8-8170-162541ac0026 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:44.609389 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701886d-f273-4633-ae94-cf4cdcd23393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:44.610711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701886d-f273-4633-ae94-cf4cdcd23393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:44.610997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701886d-f273-4633-ae94-cf4cdcd23393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:44.611283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701886d-f273-4633-ae94-cf4cdcd23393 admin admin] PATH_INFO: `/v3/projects/4d2996f0d19a4f0685bc1426b9e5f280` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:44.613330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701886d-f273-4633-ae94-cf4cdcd23393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d2996f0d19a4f0685bc1426b9e5f280)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:44.615312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701886d-f273-4633-ae94-cf4cdcd23393 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:44.620464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733236e6-a1df-4e8a-a059-0c6af561b088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:44.621749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733236e6-a1df-4e8a-a059-0c6af561b088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:44.622131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733236e6-a1df-4e8a-a059-0c6af561b088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:44.622508 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733236e6-a1df-4e8a-a059-0c6af561b088 admin admin] PATH_INFO: `/v3/projects/07ffe38046f247bb8382ddbe7acbbfce` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:44.624592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733236e6-a1df-4e8a-a059-0c6af561b088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07ffe38046f247bb8382ddbe7acbbfce)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:44.626639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733236e6-a1df-4e8a-a059-0c6af561b088 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:44.714969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-733236e6-a1df-4e8a-a059-0c6af561b088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07ffe38046f247bb8382ddbe7acbbfce', 'request_id': 'req-733236e6-a1df-4e8a-a059-0c6af561b088'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:44.715511 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6701886d-f273-4633-ae94-cf4cdcd23393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d2996f0d19a4f0685bc1426b9e5f280', 'request_id': 'req-6701886d-f273-4633-ae94-cf4cdcd23393'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:44.715630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-733236e6-a1df-4e8a-a059-0c6af561b088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07ffe38046f247bb8382ddbe7acbbfce', 'request_id': 'req-733236e6-a1df-4e8a-a059-0c6af561b088'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:44.719840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6701886d-f273-4633-ae94-cf4cdcd23393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d2996f0d19a4f0685bc1426b9e5f280', 'request_id': 'req-6701886d-f273-4633-ae94-cf4cdcd23393'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:44.727077 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-733236e6-a1df-4e8a-a059-0c6af561b088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07ffe38046f247bb8382ddbe7acbbfce', 'request_id': 'req-733236e6-a1df-4e8a-a059-0c6af561b088'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:44.735936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6701886d-f273-4633-ae94-cf4cdcd23393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d2996f0d19a4f0685bc1426b9e5f280', 'request_id': 'req-6701886d-f273-4633-ae94-cf4cdcd23393'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:44.741151 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-733236e6-a1df-4e8a-a059-0c6af561b088 admin admin] The token cache is being invalidate because project 07ffe38046f247bb8382ddbe7acbbfce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:44.741574 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-733236e6-a1df-4e8a-a059-0c6af561b088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:44.742982 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-733236e6-a1df-4e8a-a059-0c6af561b088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:44.745199 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1021/2042] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:44 2022] DELETE /identity/v3/projects/07ffe38046f247bb8382ddbe7acbbfce => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:44.752388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceec623-f390-4d70-b3c7-b3db03795dd8 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:44.752740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceec623-f390-4d70-b3c7-b3db03795dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:44.753081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceec623-f390-4d70-b3c7-b3db03795dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:44.765852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6701886d-f273-4633-ae94-cf4cdcd23393 admin admin] The token cache is being invalidate because project 4d2996f0d19a4f0685bc1426b9e5f280 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:44.766122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6701886d-f273-4633-ae94-cf4cdcd23393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:44.766999 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6701886d-f273-4633-ae94-cf4cdcd23393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:44.769584 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1022/2043] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:44 2022] DELETE /identity/v3/projects/4d2996f0d19a4f0685bc1426b9e5f280 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:44.775829 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4e869-761b-4164-9883-19aeaa25e9f8 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:44.776128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4e869-761b-4164-9883-19aeaa25e9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:44.776419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4e869-761b-4164-9883-19aeaa25e9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:44.778450 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-3ceec623-f390-4d70-b3c7-b3db03795dd8 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:44.780035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3ceec623-f390-4d70-b3c7-b3db03795dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:44.812899 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-83f4e869-761b-4164-9883-19aeaa25e9f8 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:44.815796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-83f4e869-761b-4164-9883-19aeaa25e9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:44.827424 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:44.916657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:44.965550 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1022/2044] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:44.966777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceec623-f390-4d70-b3c7-b3db03795dd8 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:44.970296 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1023/2045] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:44.974553 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0d5e0-ff76-4a91-903a-07e16be6dbf1 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:44.975675 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3ceec623-f390-4d70-b3c7-b3db03795dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:44.978952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0d5e0-ff76-4a91-903a-07e16be6dbf1 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:44.980588 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0d5e0-ff76-4a91-903a-07e16be6dbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.006428 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceec623-f390-4d70-b3c7-b3db03795dd8 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:45.011061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-49f0d5e0-ff76-4a91-903a-07e16be6dbf1 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:45.012661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-49f0d5e0-ff76-4a91-903a-07e16be6dbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:45.026446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1024/2046] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:45.028204 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75572ce5-3c74-435a-a536-dd7d1465924a None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.028649 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75572ce5-3c74-435a-a536-dd7d1465924a None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.030392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:45.031509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.031917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75572ce5-3c74-435a-a536-dd7d1465924a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.032349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.034053 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d admin admin] PATH_INFO: `/v3/projects/de36fd9508b74f948964df55faf7163e` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.035555 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d admin admin] RBAC: Authorizing `identity:delete_project(project_id=de36fd9508b74f948964df55faf7163e)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:45.037957 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:45.063926 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-75572ce5-3c74-435a-a536-dd7d1465924a None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:45.065213 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-75572ce5-3c74-435a-a536-dd7d1465924a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:45.081275 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1025/2047] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:45.082456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75572ce5-3c74-435a-a536-dd7d1465924a None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.094847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75572ce5-3c74-435a-a536-dd7d1465924a None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:45.100238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6fe35-83e1-4ebd-a352-bbeef8a6cb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:45.100923 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6fe35-83e1-4ebd-a352-bbeef8a6cb8a admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.101216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6fe35-83e1-4ebd-a352-bbeef8a6cb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.102249 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6fe35-83e1-4ebd-a352-bbeef8a6cb8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.103746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6fe35-83e1-4ebd-a352-bbeef8a6cb8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:45.104083 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6fe35-83e1-4ebd-a352-bbeef8a6cb8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:45.105929 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6fe35-83e1-4ebd-a352-bbeef8a6cb8a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:45.129019 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de36fd9508b74f948964df55faf7163e', 'request_id': 'req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:45.134732 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1026/2048] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:13:45.134732 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6fe35-83e1-4ebd-a352-bbeef8a6cb8a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.145391 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de36fd9508b74f948964df55faf7163e', 'request_id': 'req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:45.153090 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6fe35-83e1-4ebd-a352-bbeef8a6cb8a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:45.156605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de36fd9508b74f948964df55faf7163e', 'request_id': 'req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:45.181270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d admin admin] The token cache is being invalidate because project de36fd9508b74f948964df55faf7163e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:45.181573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:45.182281 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:45.184832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1023/2049] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:44 2022] DELETE /identity/v3/projects/de36fd9508b74f948964df55faf7163e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:45.185664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.188078 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:45.259396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd4375-600c-404d-909a-5b2cea44f5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:45.260359 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd4375-600c-404d-909a-5b2cea44f5de admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.260788 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd4375-600c-404d-909a-5b2cea44f5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.262454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd4375-600c-404d-909a-5b2cea44f5de admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.263398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd4375-600c-404d-909a-5b2cea44f5de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:45.263989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd4375-600c-404d-909a-5b2cea44f5de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:45.265984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd4375-600c-404d-909a-5b2cea44f5de admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:45.268559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f92c4f-bf1f-4bbe-8881-3ef69a48f83d admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:45.273681 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb8e96-9ce7-4d40-81c5-e6f2fdd9e091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:45.274772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb8e96-9ce7-4d40-81c5-e6f2fdd9e091 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.275069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb8e96-9ce7-4d40-81c5-e6f2fdd9e091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.275357 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb8e96-9ce7-4d40-81c5-e6f2fdd9e091 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.276071 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb8e96-9ce7-4d40-81c5-e6f2fdd9e091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:45.278152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb8e96-9ce7-4d40-81c5-e6f2fdd9e091 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:45.281428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1027/2050] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:13:45.283162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f0a10-bad6-4e62-ab7a-db6dd487d8eb None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.283673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f0a10-bad6-4e62-ab7a-db6dd487d8eb None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.284291 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f0a10-bad6-4e62-ab7a-db6dd487d8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.311782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-515f0a10-bad6-4e62-ab7a-db6dd487d8eb None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:45.313015 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-515f0a10-bad6-4e62-ab7a-db6dd487d8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:45.322375 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1024/2051] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:45 2022] POST /identity/v3/projects => generated 377 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:45.323535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb8e96-9ce7-4d40-81c5-e6f2fdd9e091 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.326458 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-69bb8e96-9ce7-4d40-81c5-e6f2fdd9e091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:45.330733 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1028/2052] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:45.331705 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f0a10-bad6-4e62-ab7a-db6dd487d8eb None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.345090 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f0a10-bad6-4e62-ab7a-db6dd487d8eb None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:45.409398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:45.428740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb8e96-9ce7-4d40-81c5-e6f2fdd9e091 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:45.437391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f702f-0fbc-45d7-8ad3-7d1ba16c8920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:45.438035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f702f-0fbc-45d7-8ad3-7d1ba16c8920 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.438325 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f702f-0fbc-45d7-8ad3-7d1ba16c8920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.438717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f702f-0fbc-45d7-8ad3-7d1ba16c8920 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.439425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f702f-0fbc-45d7-8ad3-7d1ba16c8920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:45.440902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef33045-1694-493e-a9be-4561566333f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:45.441459 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f702f-0fbc-45d7-8ad3-7d1ba16c8920 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:45.441827 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef33045-1694-493e-a9be-4561566333f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.442186 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef33045-1694-493e-a9be-4561566333f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.442652 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef33045-1694-493e-a9be-4561566333f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.443470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef33045-1694-493e-a9be-4561566333f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:45.445594 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef33045-1694-493e-a9be-4561566333f4 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:45.461885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:45.469818 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1029/2053] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:45 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:45.472150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fe605-3b9b-41e6-b4c2-ee7048057f4b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.474462 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fe605-3b9b-41e6-b4c2-ee7048057f4b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.474614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fe605-3b9b-41e6-b4c2-ee7048057f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.498829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1025/2054] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:45 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:45.498829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef33045-1694-493e-a9be-4561566333f4 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.500404 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-2f8fe605-3b9b-41e6-b4c2-ee7048057f4b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:45.501682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2f8fe605-3b9b-41e6-b4c2-ee7048057f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:45.576678 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1030/2055] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:45.577410 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fe605-3b9b-41e6-b4c2-ee7048057f4b None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.588436 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fe605-3b9b-41e6-b4c2-ee7048057f4b None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:45.594369 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f216b4-c7c4-41c7-805e-5ad56d4c0237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:45.595079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f216b4-c7c4-41c7-805e-5ad56d4c0237 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.595392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f216b4-c7c4-41c7-805e-5ad56d4c0237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.595683 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f216b4-c7c4-41c7-805e-5ad56d4c0237 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.595972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef33045-1694-493e-a9be-4561566333f4 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:45.596387 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f216b4-c7c4-41c7-805e-5ad56d4c0237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:45.598474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f216b4-c7c4-41c7-805e-5ad56d4c0237 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:45.600797 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85f227-a3b7-4380-aa21-ee4be7809607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:45.601542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85f227-a3b7-4380-aa21-ee4be7809607 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.601829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85f227-a3b7-4380-aa21-ee4be7809607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.602109 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85f227-a3b7-4380-aa21-ee4be7809607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.602874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd85f227-a3b7-4380-aa21-ee4be7809607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:45.604657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd85f227-a3b7-4380-aa21-ee4be7809607 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:45.620374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1026/2056] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:45.620374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85f227-a3b7-4380-aa21-ee4be7809607 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.620567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1031/2057] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:45 2022] POST /identity/v3/users => generated 292 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:45.621276 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f216b4-c7c4-41c7-805e-5ad56d4c0237 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.632268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85f227-a3b7-4380-aa21-ee4be7809607 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:45.633285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f216b4-c7c4-41c7-805e-5ad56d4c0237 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:45.637810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a970759-2b7d-4b20-9f9b-17fea8a453e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:45.638771 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f434b01-e10a-473c-85dc-0ef791c2a6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:45.638918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a970759-2b7d-4b20-9f9b-17fea8a453e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.639087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a970759-2b7d-4b20-9f9b-17fea8a453e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.639470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a970759-2b7d-4b20-9f9b-17fea8a453e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.639587 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f434b01-e10a-473c-85dc-0ef791c2a6e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.639865 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f434b01-e10a-473c-85dc-0ef791c2a6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.640153 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f434b01-e10a-473c-85dc-0ef791c2a6e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.640732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a970759-2b7d-4b20-9f9b-17fea8a453e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:45.640895 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f434b01-e10a-473c-85dc-0ef791c2a6e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:45.641157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a970759-2b7d-4b20-9f9b-17fea8a453e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:45.642616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f434b01-e10a-473c-85dc-0ef791c2a6e2 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:45.643002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a970759-2b7d-4b20-9f9b-17fea8a453e9 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:45.657268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1027/2058] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:13:45.658243 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a970759-2b7d-4b20-9f9b-17fea8a453e9 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.659838 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1032/2059] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:45.661273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f434b01-e10a-473c-85dc-0ef791c2a6e2 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.672957 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a970759-2b7d-4b20-9f9b-17fea8a453e9 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:45.673201 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f434b01-e10a-473c-85dc-0ef791c2a6e2 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:45.677735 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630125d8-1c1b-4e7d-ae70-65b56f4aa826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:45.678389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630125d8-1c1b-4e7d-ae70-65b56f4aa826 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.678672 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630125d8-1c1b-4e7d-ae70-65b56f4aa826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.679008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630125d8-1c1b-4e7d-ae70-65b56f4aa826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.680429 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630125d8-1c1b-4e7d-ae70-65b56f4aa826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:45.681667 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5776e4d-f902-496b-8bf8-693b943cfd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:45.682164 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630125d8-1c1b-4e7d-ae70-65b56f4aa826 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:45.683074 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5776e4d-f902-496b-8bf8-693b943cfd42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.683683 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5776e4d-f902-496b-8bf8-693b943cfd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.684051 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5776e4d-f902-496b-8bf8-693b943cfd42 admin admin] PATH_INFO: `/v3/projects/cbaca913aca2411cbbade4fd5236c0a1/users/394780314b1b4baf8a0d7405ed69783f/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.698388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1028/2060] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:45.698388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630125d8-1c1b-4e7d-ae70-65b56f4aa826 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.705912 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630125d8-1c1b-4e7d-ae70-65b56f4aa826 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:45.706449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5776e4d-f902-496b-8bf8-693b943cfd42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbaca913aca2411cbbade4fd5236c0a1, user_id=394780314b1b4baf8a0d7405ed69783f, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:45.709502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5776e4d-f902-496b-8bf8-693b943cfd42 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:45.711511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c657d7-73eb-4a2d-bc68-e3c20a867fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:45.712285 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c657d7-73eb-4a2d-bc68-e3c20a867fee admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.712646 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c657d7-73eb-4a2d-bc68-e3c20a867fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.712989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c657d7-73eb-4a2d-bc68-e3c20a867fee admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.713761 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c657d7-73eb-4a2d-bc68-e3c20a867fee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:45.716288 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c657d7-73eb-4a2d-bc68-e3c20a867fee admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:45.738020 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1033/2061] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:45 2022] PUT /identity/v3/projects/cbaca913aca2411cbbade4fd5236c0a1/users/394780314b1b4baf8a0d7405ed69783f/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:45.738020 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5776e4d-f902-496b-8bf8-693b943cfd42 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.751922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1029/2062] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:45 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:45.751922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edc650-3c2d-45f0-ab08-314402257d60 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.751922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edc650-3c2d-45f0-ab08-314402257d60 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.751922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edc650-3c2d-45f0-ab08-314402257d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.782787 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-86edc650-3c2d-45f0-ab08-314402257d60 None None] MFA Rules not processed for user `394780314b1b4baf8a0d7405ed69783f`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:45.784011 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-86edc650-3c2d-45f0-ab08-314402257d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:45.818196 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5776e4d-f902-496b-8bf8-693b943cfd42 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:45.870848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1030/2063] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:45.871783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edc650-3c2d-45f0-ab08-314402257d60 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.889103 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:45.897174 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:45.902538 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1eed7-57dc-40b2-bf78-f3273a169ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:45.903282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1eed7-57dc-40b2-bf78-f3273a169ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.903570 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1eed7-57dc-40b2-bf78-f3273a169ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.904378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1eed7-57dc-40b2-bf78-f3273a169ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.905287 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1eed7-57dc-40b2-bf78-f3273a169ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:45.907094 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1eed7-57dc-40b2-bf78-f3273a169ed1 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:45.924839 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1034/2064] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:45.928992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1eed7-57dc-40b2-bf78-f3273a169ed1 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.940495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1eed7-57dc-40b2-bf78-f3273a169ed1 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:45.945325 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e199cb-0319-41cc-8649-ad310cf22265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:45.946247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e199cb-0319-41cc-8649-ad310cf22265 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.946601 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e199cb-0319-41cc-8649-ad310cf22265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.947040 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e199cb-0319-41cc-8649-ad310cf22265 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.947747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e199cb-0319-41cc-8649-ad310cf22265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:45.949955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e199cb-0319-41cc-8649-ad310cf22265 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:45.952714 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edc650-3c2d-45f0-ab08-314402257d60 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:45.957533 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66812dcf-f147-4695-a376-40d487044fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:45.958300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66812dcf-f147-4695-a376-40d487044fdd admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:45.958655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66812dcf-f147-4695-a376-40d487044fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:45.959061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66812dcf-f147-4695-a376-40d487044fdd admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:45.959837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66812dcf-f147-4695-a376-40d487044fdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:45.962420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66812dcf-f147-4695-a376-40d487044fdd admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:45.981084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1035/2065] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:45 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:45.981993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e199cb-0319-41cc-8649-ad310cf22265 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.988117 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1031/2066] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:45 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:45.988117 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66812dcf-f147-4695-a376-40d487044fdd admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:45.998451 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66812dcf-f147-4695-a376-40d487044fdd admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:46.073358 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e199cb-0319-41cc-8649-ad310cf22265 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:46.073550 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e12b0-d76c-4ae4-9e21-46526606d7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:46.074077 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e12b0-d76c-4ae4-9e21-46526606d7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:46.074349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e12b0-d76c-4ae4-9e21-46526606d7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:46.074624 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e12b0-d76c-4ae4-9e21-46526606d7a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:46.075415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e12b0-d76c-4ae4-9e21-46526606d7a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:46.077345 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e12b0-d76c-4ae4-9e21-46526606d7a9 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:46.078162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd55f3-98f2-49df-8cb9-739c3c065e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:46.079085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd55f3-98f2-49df-8cb9-739c3c065e11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:46.079372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd55f3-98f2-49df-8cb9-739c3c065e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:46.079652 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd55f3-98f2-49df-8cb9-739c3c065e11 admin admin] PATH_INFO: `/v3/projects/5289f735719544f2a758895a6d4bf3c0/users/641ade550b8846efb1aff2adb151ac89/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:46.096261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd55f3-98f2-49df-8cb9-739c3c065e11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5289f735719544f2a758895a6d4bf3c0, user_id=641ade550b8846efb1aff2adb151ac89, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:46.098943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd55f3-98f2-49df-8cb9-739c3c065e11 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:46.101494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1032/2067] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:45 2022] POST /identity/v3/users => generated 297 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:46.102268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e12b0-d76c-4ae4-9e21-46526606d7a9 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:46.113690 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e12b0-d76c-4ae4-9e21-46526606d7a9 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:46.116649 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1036/2068] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:45 2022] PUT /identity/v3/projects/5289f735719544f2a758895a6d4bf3c0/users/641ade550b8846efb1aff2adb151ac89/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:46.118315 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd55f3-98f2-49df-8cb9-739c3c065e11 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:46.120255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ffcd55f3-98f2-49df-8cb9-739c3c065e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:46.187008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932ad43-365c-4270-b8e8-8a493f227e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:46.188162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932ad43-365c-4270-b8e8-8a493f227e41 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:46.188479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932ad43-365c-4270-b8e8-8a493f227e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:46.188895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932ad43-365c-4270-b8e8-8a493f227e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:46.189621 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932ad43-365c-4270-b8e8-8a493f227e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:46.192521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932ad43-365c-4270-b8e8-8a493f227e41 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:46.200650 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd55f3-98f2-49df-8cb9-739c3c065e11 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:46.205662 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bbe1d-c28a-47d6-9bbc-c9be74c72528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:46.206608 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bbe1d-c28a-47d6-9bbc-c9be74c72528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:46.206941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bbe1d-c28a-47d6-9bbc-c9be74c72528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:46.207223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bbe1d-c28a-47d6-9bbc-c9be74c72528 admin admin] PATH_INFO: `/v3/users/ad43df748f09486181a0cc764fb004b1` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:46.208451 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1bbe1d-c28a-47d6-9bbc-c9be74c72528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad43df748f09486181a0cc764fb004b1)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:46.210577 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1bbe1d-c28a-47d6-9bbc-c9be74c72528 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:46.222728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1033/2069] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:46.222728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932ad43-365c-4270-b8e8-8a493f227e41 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:46.234546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932ad43-365c-4270-b8e8-8a493f227e41 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:46.242977 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e2764-6205-4298-b6d2-edc364291bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:46.244397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e2764-6205-4298-b6d2-edc364291bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:46.244397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e2764-6205-4298-b6d2-edc364291bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:46.244612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e2764-6205-4298-b6d2-edc364291bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:46.245296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e2764-6205-4298-b6d2-edc364291bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:46.247034 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e2764-6205-4298-b6d2-edc364291bf1 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:46.258766 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1034/2070] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:46.263472 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848f789-611f-431e-8afc-f4243349d514 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:46.263863 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848f789-611f-431e-8afc-f4243349d514 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:46.264308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848f789-611f-431e-8afc-f4243349d514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:46.280667 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-9e1bbe1d-c28a-47d6-9bbc-c9be74c72528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad43df748f09486181a0cc764fb004b1'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:46.286638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:46.298224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-9e1bbe1d-c28a-47d6-9bbc-c9be74c72528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad43df748f09486181a0cc764fb004b1'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:46.302106 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-b848f789-611f-431e-8afc-f4243349d514 None None] MFA Rules not processed for user `641ade550b8846efb1aff2adb151ac89`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:46.303779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b848f789-611f-431e-8afc-f4243349d514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:46.307532 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-9e1bbe1d-c28a-47d6-9bbc-c9be74c72528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad43df748f09486181a0cc764fb004b1'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:46.308274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-9e1bbe1d-c28a-47d6-9bbc-c9be74c72528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad43df748f09486181a0cc764fb004b1'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:46.319982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-9e1bbe1d-c28a-47d6-9bbc-c9be74c72528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad43df748f09486181a0cc764fb004b1'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:46.323052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:46.328889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1037/2071] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:46 2022] DELETE /identity/v3/users/ad43df748f09486181a0cc764fb004b1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:46.329592 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bbe1d-c28a-47d6-9bbc-c9be74c72528 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:46.331168 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9e1bbe1d-c28a-47d6-9bbc-c9be74c72528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:46.398536 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1035/2072] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:46.398768 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bbe1d-c28a-47d6-9bbc-c9be74c72528 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:46.399741 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848f789-611f-431e-8afc-f4243349d514 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:46.401051 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b848f789-611f-431e-8afc-f4243349d514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:46.403805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774d346-f331-4b3e-96f4-37493c4fdd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:46.404448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774d346-f331-4b3e-96f4-37493c4fdd48 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:46.404735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774d346-f331-4b3e-96f4-37493c4fdd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:46.404973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774d346-f331-4b3e-96f4-37493c4fdd48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:46.405706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774d346-f331-4b3e-96f4-37493c4fdd48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:46.407322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774d346-f331-4b3e-96f4-37493c4fdd48 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:46.416831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848f789-611f-431e-8afc-f4243349d514 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:46.421278 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1038/2073] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:46.421278 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a046bd-6063-4201-962c-75dd7bf62acc None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:46.421278 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a046bd-6063-4201-962c-75dd7bf62acc None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:46.421278 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a046bd-6063-4201-962c-75dd7bf62acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:46.423281 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6da18-698f-4150-bff2-d2b296f903fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:46.424171 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6da18-698f-4150-bff2-d2b296f903fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:46.424457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6da18-698f-4150-bff2-d2b296f903fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:46.424740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6da18-698f-4150-bff2-d2b296f903fe admin admin] PATH_INFO: `/v3/projects/5dea97c414ce4acf907e3b525774ead4/users/84bbd2eecd474802af33f1f5155590c1/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:46.437860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-64a046bd-6063-4201-962c-75dd7bf62acc None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:46.439085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-64a046bd-6063-4201-962c-75dd7bf62acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:46.440990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6da18-698f-4150-bff2-d2b296f903fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dea97c414ce4acf907e3b525774ead4, user_id=84bbd2eecd474802af33f1f5155590c1, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:46.443734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6da18-698f-4150-bff2-d2b296f903fe admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:46.446856 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1039/2074] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:46.450877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a046bd-6063-4201-962c-75dd7bf62acc None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:46.450877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-64a046bd-6063-4201-962c-75dd7bf62acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:46.462357 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1036/2075] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:46 2022] PUT /identity/v3/projects/5dea97c414ce4acf907e3b525774ead4/users/84bbd2eecd474802af33f1f5155590c1/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:46.463254 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6da18-698f-4150-bff2-d2b296f903fe admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:46.509748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a046bd-6063-4201-962c-75dd7bf62acc None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:46.526896 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6da18-698f-4150-bff2-d2b296f903fe admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:46.531823 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867f075-8788-47f8-885d-be2b836a15ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:46.532708 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867f075-8788-47f8-885d-be2b836a15ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:46.532988 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867f075-8788-47f8-885d-be2b836a15ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:46.533264 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867f075-8788-47f8-885d-be2b836a15ae admin admin] PATH_INFO: `/v3/projects/3d9aa408366d452fac34f7bd1d17f79e/users/1ca9be4fcbaa472396d9b51206a73321/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:46.548297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2867f075-8788-47f8-885d-be2b836a15ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d9aa408366d452fac34f7bd1d17f79e, user_id=1ca9be4fcbaa472396d9b51206a73321, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:46.551016 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2867f075-8788-47f8-885d-be2b836a15ae admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:46.562996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ea526-84fd-48d1-979e-3f1b88926df3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:46.563777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ea526-84fd-48d1-979e-3f1b88926df3 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:46.564062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ea526-84fd-48d1-979e-3f1b88926df3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:46.564340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ea526-84fd-48d1-979e-3f1b88926df3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:46.565106 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1037/2076] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:46 2022] PUT /identity/v3/projects/3d9aa408366d452fac34f7bd1d17f79e/users/1ca9be4fcbaa472396d9b51206a73321/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:46.565927 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867f075-8788-47f8-885d-be2b836a15ae admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:46.567519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-2867f075-8788-47f8-885d-be2b836a15ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:46.601386 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:46.631752 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867f075-8788-47f8-885d-be2b836a15ae admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:46.631986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ea526-84fd-48d1-979e-3f1b88926df3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:46.633697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ea526-84fd-48d1-979e-3f1b88926df3 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:46.635515 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23dba14-c642-4719-a48f-23cd70db0e2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:46.636161 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dba14-c642-4719-a48f-23cd70db0e2d service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:46.636449 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dba14-c642-4719-a48f-23cd70db0e2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:46.636731 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23dba14-c642-4719-a48f-23cd70db0e2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:46.642818 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1040/2077] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:46.643619 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ea526-84fd-48d1-979e-3f1b88926df3 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:46.655400 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ea526-84fd-48d1-979e-3f1b88926df3 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:46.662990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:46.704819 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dba14-c642-4719-a48f-23cd70db0e2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:46.706191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23dba14-c642-4719-a48f-23cd70db0e2d service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:46.716045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4949f6-68ed-4f05-9843-877e73cd998b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:46.717344 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4949f6-68ed-4f05-9843-877e73cd998b admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:46.717344 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4949f6-68ed-4f05-9843-877e73cd998b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:46.717541 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1038/2078] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:46.717998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4949f6-68ed-4f05-9843-877e73cd998b admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:46.718368 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4949f6-68ed-4f05-9843-877e73cd998b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:46.718477 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a29a0-5d86-4b6a-9706-7253a55b049b None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:46.718734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a29a0-5d86-4b6a-9706-7253a55b049b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:46.719018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a29a0-5d86-4b6a-9706-7253a55b049b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:46.719997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4949f6-68ed-4f05-9843-877e73cd998b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:46.733636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1041/2079] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:46.737739 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4949f6-68ed-4f05-9843-877e73cd998b admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:46.750746 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-c33a29a0-5d86-4b6a-9706-7253a55b049b None None] MFA Rules not processed for user `1ca9be4fcbaa472396d9b51206a73321`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:46.751812 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4949f6-68ed-4f05-9843-877e73cd998b admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:46.753377 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c33a29a0-5d86-4b6a-9706-7253a55b049b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:46.756876 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81584018-8fb1-4b54-b0b7-900091cba733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:46.758539 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81584018-8fb1-4b54-b0b7-900091cba733 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:46.758857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81584018-8fb1-4b54-b0b7-900091cba733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:46.759140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81584018-8fb1-4b54-b0b7-900091cba733 admin admin] PATH_INFO: `/v3/projects/5dea97c414ce4acf907e3b525774ead4/users/84bbd2eecd474802af33f1f5155590c1/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:46.761485 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81584018-8fb1-4b54-b0b7-900091cba733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dea97c414ce4acf907e3b525774ead4, user_id=84bbd2eecd474802af33f1f5155590c1, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:46.765262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81584018-8fb1-4b54-b0b7-900091cba733 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:46.779547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1042/2080] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:46 2022] PUT /identity/v3/projects/5dea97c414ce4acf907e3b525774ead4/users/84bbd2eecd474802af33f1f5155590c1/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:46.789899 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea73ddd-06af-4a61-824a-c6b14b1b114b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:46.790202 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea73ddd-06af-4a61-824a-c6b14b1b114b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:46.790478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea73ddd-06af-4a61-824a-c6b14b1b114b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:46.832545 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-5ea73ddd-06af-4a61-824a-c6b14b1b114b None None] MFA Rules not processed for user `84bbd2eecd474802af33f1f5155590c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:46.834206 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5ea73ddd-06af-4a61-824a-c6b14b1b114b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:46.870336 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1039/2081] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:46.878510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a29a0-5d86-4b6a-9706-7253a55b049b None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:46.880563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c33a29a0-5d86-4b6a-9706-7253a55b049b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:46.973018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a29a0-5d86-4b6a-9706-7253a55b049b None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:46.998109 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1043/2082] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:47.039572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480060b1-ef46-4a82-aad7-05a642c9468a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:47.040346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480060b1-ef46-4a82-aad7-05a642c9468a service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:47.040589 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480060b1-ef46-4a82-aad7-05a642c9468a service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:47.041410 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480060b1-ef46-4a82-aad7-05a642c9468a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:47.081852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:47.093128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea73ddd-06af-4a61-824a-c6b14b1b114b None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:47.095178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5ea73ddd-06af-4a61-824a-c6b14b1b114b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:47.133934 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480060b1-ef46-4a82-aad7-05a642c9468a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:47.135832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480060b1-ef46-4a82-aad7-05a642c9468a service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:47.147654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1040/2083] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:47.151712 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480060b1-ef46-4a82-aad7-05a642c9468a service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:47.161672 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480060b1-ef46-4a82-aad7-05a642c9468a service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:47.165160 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:47.166870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2129f-d33d-4048-924e-d1b0c45c9fd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:47.167304 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2129f-d33d-4048-924e-d1b0c45c9fd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:47.167629 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2129f-d33d-4048-924e-d1b0c45c9fd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:47.167918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2129f-d33d-4048-924e-d1b0c45c9fd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:47.179367 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2129f-d33d-4048-924e-d1b0c45c9fd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:47.181177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2129f-d33d-4048-924e-d1b0c45c9fd0 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:47.191545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1041/2084] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:47.192647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea73ddd-06af-4a61-824a-c6b14b1b114b None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:47.202201 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1874dc0-9d22-423b-8917-3bfa0a3706d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:47.203111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1874dc0-9d22-423b-8917-3bfa0a3706d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:47.203391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1874dc0-9d22-423b-8917-3bfa0a3706d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:47.203695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1874dc0-9d22-423b-8917-3bfa0a3706d4 admin admin] PATH_INFO: `/v3/projects/346ff0b059fa4bc2b4b8426ff8ba71bf` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:47.205004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1874dc0-9d22-423b-8917-3bfa0a3706d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=346ff0b059fa4bc2b4b8426ff8ba71bf)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:47.207106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1874dc0-9d22-423b-8917-3bfa0a3706d4 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:47.263100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-f1874dc0-9d22-423b-8917-3bfa0a3706d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '346ff0b059fa4bc2b4b8426ff8ba71bf', 'request_id': 'req-f1874dc0-9d22-423b-8917-3bfa0a3706d4'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:47.269161 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-f1874dc0-9d22-423b-8917-3bfa0a3706d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '346ff0b059fa4bc2b4b8426ff8ba71bf', 'request_id': 'req-f1874dc0-9d22-423b-8917-3bfa0a3706d4'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:47.276071 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-f1874dc0-9d22-423b-8917-3bfa0a3706d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '346ff0b059fa4bc2b4b8426ff8ba71bf', 'request_id': 'req-f1874dc0-9d22-423b-8917-3bfa0a3706d4'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:47.293113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-f1874dc0-9d22-423b-8917-3bfa0a3706d4 admin admin] The token cache is being invalidate because project 346ff0b059fa4bc2b4b8426ff8ba71bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:47.293531 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-f1874dc0-9d22-423b-8917-3bfa0a3706d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:47.294187 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-f1874dc0-9d22-423b-8917-3bfa0a3706d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:47.295814 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1044/2085] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:47 2022] DELETE /identity/v3/projects/346ff0b059fa4bc2b4b8426ff8ba71bf => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:47.300092 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2129f-d33d-4048-924e-d1b0c45c9fd0 service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:47.314305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2129f-d33d-4048-924e-d1b0c45c9fd0 service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:47.412312 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc3dd0-4832-4844-b401-fa870526b6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:47.415499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc3dd0-4832-4844-b401-fa870526b6bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:47.416084 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc3dd0-4832-4844-b401-fa870526b6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:47.416447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc3dd0-4832-4844-b401-fa870526b6bd admin admin] PATH_INFO: `/v3/users/f021ca5d232f4c3e9ef9ee6ed86b5744` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:47.418056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc3dd0-4832-4844-b401-fa870526b6bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f021ca5d232f4c3e9ef9ee6ed86b5744)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:47.420544 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc3dd0-4832-4844-b401-fa870526b6bd admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:47.513083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-61bc3dd0-4832-4844-b401-fa870526b6bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f021ca5d232f4c3e9ef9ee6ed86b5744'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:47.532399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-61bc3dd0-4832-4844-b401-fa870526b6bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f021ca5d232f4c3e9ef9ee6ed86b5744'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:47.545958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-61bc3dd0-4832-4844-b401-fa870526b6bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f021ca5d232f4c3e9ef9ee6ed86b5744'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:47.545958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-61bc3dd0-4832-4844-b401-fa870526b6bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f021ca5d232f4c3e9ef9ee6ed86b5744'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:47.551071 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-61bc3dd0-4832-4844-b401-fa870526b6bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f021ca5d232f4c3e9ef9ee6ed86b5744'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:47.561100 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1042/2086] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:47 2022] DELETE /identity/v3/users/f021ca5d232f4c3e9ef9ee6ed86b5744 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:47.620505 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1874dc0-9d22-423b-8917-3bfa0a3706d4 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:47.622087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f1874dc0-9d22-423b-8917-3bfa0a3706d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:47.693712 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1874dc0-9d22-423b-8917-3bfa0a3706d4 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:47.761355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f16979-9900-4bc5-b5d5-57d2ac168734 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84bbd2eecd474802af33f1f5155590c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dea97c414ce4acf907e3b525774ead4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:47.762043 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f16979-9900-4bc5-b5d5-57d2ac168734 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:47.762335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f16979-9900-4bc5-b5d5-57d2ac168734 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:47.762629 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f16979-9900-4bc5-b5d5-57d2ac168734 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:47.763401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f16979-9900-4bc5-b5d5-57d2ac168734 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:47.763719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f16979-9900-4bc5-b5d5-57d2ac168734 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:47.765370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f16979-9900-4bc5-b5d5-57d2ac168734 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:47.778307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1045/2087] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:13:47.782484 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc3dd0-4832-4844-b401-fa870526b6bd admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:47.783944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-61bc3dd0-4832-4844-b401-fa870526b6bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:47.797634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc3dd0-4832-4844-b401-fa870526b6bd admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:47.802135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e7feb-214f-4b4e-8caa-d87c491f53b2 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84bbd2eecd474802af33f1f5155590c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dea97c414ce4acf907e3b525774ead4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:47.802830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e7feb-214f-4b4e-8caa-d87c491f53b2 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:47.803123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e7feb-214f-4b4e-8caa-d87c491f53b2 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:47.803410 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e7feb-214f-4b4e-8caa-d87c491f53b2 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:47.804431 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e7feb-214f-4b4e-8caa-d87c491f53b2 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:47.806074 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e7feb-214f-4b4e-8caa-d87c491f53b2 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:47.833156 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1043/2088] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:47 2022] POST /identity/v3/projects => generated 353 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:47.837892 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f16979-9900-4bc5-b5d5-57d2ac168734 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:47.845342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e7feb-214f-4b4e-8caa-d87c491f53b2 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:47.847231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e03e7feb-214f-4b4e-8caa-d87c491f53b2 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:47.849552 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f16979-9900-4bc5-b5d5-57d2ac168734 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:47.855265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:47.863941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:47.925305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15daa91-4405-4c18-9807-6bc1b68cb6b3 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84bbd2eecd474802af33f1f5155590c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dea97c414ce4acf907e3b525774ead4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:47.925985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15daa91-4405-4c18-9807-6bc1b68cb6b3 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:47.926279 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15daa91-4405-4c18-9807-6bc1b68cb6b3 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:47.926567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15daa91-4405-4c18-9807-6bc1b68cb6b3 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:47.927383 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15daa91-4405-4c18-9807-6bc1b68cb6b3 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:47.927701 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15daa91-4405-4c18-9807-6bc1b68cb6b3 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:47.929286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15daa91-4405-4c18-9807-6bc1b68cb6b3 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:47.935792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e7feb-214f-4b4e-8caa-d87c491f53b2 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:47.941169 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f952df-f5c5-4752-9bdb-545c680bb0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:47.941386 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1046/2089] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:13:47.942344 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f952df-f5c5-4752-9bdb-545c680bb0bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:47.942633 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f952df-f5c5-4752-9bdb-545c680bb0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:47.942946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f952df-f5c5-4752-9bdb-545c680bb0bb admin admin] PATH_INFO: `/v3/projects/bed878d897ea4974b46535b0d11cf26b` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:47.944175 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f952df-f5c5-4752-9bdb-545c680bb0bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bed878d897ea4974b46535b0d11cf26b)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:47.946624 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f952df-f5c5-4752-9bdb-545c680bb0bb admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:47.998457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-34f952df-f5c5-4752-9bdb-545c680bb0bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bed878d897ea4974b46535b0d11cf26b', 'request_id': 'req-34f952df-f5c5-4752-9bdb-545c680bb0bb'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:47.998612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15daa91-4405-4c18-9807-6bc1b68cb6b3 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:48.004109 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-34f952df-f5c5-4752-9bdb-545c680bb0bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bed878d897ea4974b46535b0d11cf26b', 'request_id': 'req-34f952df-f5c5-4752-9bdb-545c680bb0bb'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:48.009592 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15daa91-4405-4c18-9807-6bc1b68cb6b3 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:48.012405 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-34f952df-f5c5-4752-9bdb-545c680bb0bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bed878d897ea4974b46535b0d11cf26b', 'request_id': 'req-34f952df-f5c5-4752-9bdb-545c680bb0bb'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:48.024169 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-34f952df-f5c5-4752-9bdb-545c680bb0bb admin admin] The token cache is being invalidate because project bed878d897ea4974b46535b0d11cf26b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:48.024453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-34f952df-f5c5-4752-9bdb-545c680bb0bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:48.025133 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-34f952df-f5c5-4752-9bdb-545c680bb0bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:48.026700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1044/2090] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:47 2022] DELETE /identity/v3/projects/bed878d897ea4974b46535b0d11cf26b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:48.031845 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48713227-2b26-4553-9476-f705d737b658 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:48.032422 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48713227-2b26-4553-9476-f705d737b658 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:48.032707 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48713227-2b26-4553-9476-f705d737b658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:48.052048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-48713227-2b26-4553-9476-f705d737b658 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:48.053247 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-48713227-2b26-4553-9476-f705d737b658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:48.073418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b259fc-2fe6-4860-8f6f-a98b86585b14 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84bbd2eecd474802af33f1f5155590c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5dea97c414ce4acf907e3b525774ead4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:48.074303 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b259fc-2fe6-4860-8f6f-a98b86585b14 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:48.074591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b259fc-2fe6-4860-8f6f-a98b86585b14 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:48.076321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b259fc-2fe6-4860-8f6f-a98b86585b14 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] PATH_INFO: `/v3/projects/4632b62c101e4b9295c2c882f235b395` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:48.078884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b259fc-2fe6-4860-8f6f-a98b86585b14 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4632b62c101e4b9295c2c882f235b395)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:48.080809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b259fc-2fe6-4860-8f6f-a98b86585b14 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:48.134649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1045/2091] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:48.136871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63c9ad-0231-4412-add1-166632fdd051 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:48.137295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63c9ad-0231-4412-add1-166632fdd051 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:48.137848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63c9ad-0231-4412-add1-166632fdd051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:48.140123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-a7b259fc-2fe6-4860-8f6f-a98b86585b14 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4632b62c101e4b9295c2c882f235b395', 'request_id': 'req-a7b259fc-2fe6-4860-8f6f-a98b86585b14'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:48.150955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-a7b259fc-2fe6-4860-8f6f-a98b86585b14 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4632b62c101e4b9295c2c882f235b395', 'request_id': 'req-a7b259fc-2fe6-4860-8f6f-a98b86585b14'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:48.158654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-a7b259fc-2fe6-4860-8f6f-a98b86585b14 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4632b62c101e4b9295c2c882f235b395', 'request_id': 'req-a7b259fc-2fe6-4860-8f6f-a98b86585b14'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:48.160310 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-0e63c9ad-0231-4412-add1-166632fdd051 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:48.161518 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0e63c9ad-0231-4412-add1-166632fdd051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:48.180338 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-a7b259fc-2fe6-4860-8f6f-a98b86585b14 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] The token cache is being invalidate because project 4632b62c101e4b9295c2c882f235b395 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:48.180741 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-a7b259fc-2fe6-4860-8f6f-a98b86585b14 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:48.181437 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-a7b259fc-2fe6-4860-8f6f-a98b86585b14 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:48.186348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1047/2092] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:47 2022] DELETE /identity/v3/projects/4632b62c101e4b9295c2c882f235b395 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:48.197825 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:48.235180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1046/2093] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:48.240034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b259fc-2fe6-4860-8f6f-a98b86585b14 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:48.252822 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b259fc-2fe6-4860-8f6f-a98b86585b14 tempest-VolumeQuotaClassesTest-2051698585 tempest-VolumeQuotaClassesTest-2051698585-project-admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:48.259430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14821d82-cfe0-43df-a3fd-df0543a468fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:48.260219 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14821d82-cfe0-43df-a3fd-df0543a468fc admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:48.260468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14821d82-cfe0-43df-a3fd-df0543a468fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:48.260819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14821d82-cfe0-43df-a3fd-df0543a468fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:48.261635 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14821d82-cfe0-43df-a3fd-df0543a468fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:48.261940 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14821d82-cfe0-43df-a3fd-df0543a468fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:48.263976 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14821d82-cfe0-43df-a3fd-df0543a468fc admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:48.280308 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1048/2094] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:13:48.284951 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63c9ad-0231-4412-add1-166632fdd051 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:48.295619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63c9ad-0231-4412-add1-166632fdd051 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:48.300767 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4541aa7-03f3-49de-945e-4cc115978012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:48.301506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4541aa7-03f3-49de-945e-4cc115978012 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:48.301856 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4541aa7-03f3-49de-945e-4cc115978012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:48.302169 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4541aa7-03f3-49de-945e-4cc115978012 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:48.302912 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4541aa7-03f3-49de-945e-4cc115978012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:48.305578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4541aa7-03f3-49de-945e-4cc115978012 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:48.340518 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1047/2095] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:48 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:48.344482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14821d82-cfe0-43df-a3fd-df0543a468fc admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:48.356360 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14821d82-cfe0-43df-a3fd-df0543a468fc admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:48.394516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:48.411682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4541aa7-03f3-49de-945e-4cc115978012 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:48.413468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b4541aa7-03f3-49de-945e-4cc115978012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:48.437916 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05201b84-01c6-4f5c-9843-75dcb81af97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:48.438798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05201b84-01c6-4f5c-9843-75dcb81af97e admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:48.439230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05201b84-01c6-4f5c-9843-75dcb81af97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:48.439516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05201b84-01c6-4f5c-9843-75dcb81af97e admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:48.440273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05201b84-01c6-4f5c-9843-75dcb81af97e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:48.442378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05201b84-01c6-4f5c-9843-75dcb81af97e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:48.463790 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1049/2096] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:48 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:48.468118 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05201b84-01c6-4f5c-9843-75dcb81af97e admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:48.478754 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05201b84-01c6-4f5c-9843-75dcb81af97e admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:48.482487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4541aa7-03f3-49de-945e-4cc115978012 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:48.483620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0290cbd-7000-4f0a-aa99-8d0345713c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:48.484349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0290cbd-7000-4f0a-aa99-8d0345713c8b admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:48.484732 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0290cbd-7000-4f0a-aa99-8d0345713c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:48.485044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0290cbd-7000-4f0a-aa99-8d0345713c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:48.485896 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0290cbd-7000-4f0a-aa99-8d0345713c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:48.487161 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cb7bd-3bf6-4ac5-9521-e382da16d41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:48.487483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0290cbd-7000-4f0a-aa99-8d0345713c8b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:48.488027 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cb7bd-3bf6-4ac5-9521-e382da16d41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:48.488307 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cb7bd-3bf6-4ac5-9521-e382da16d41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:48.488586 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cb7bd-3bf6-4ac5-9521-e382da16d41c admin admin] PATH_INFO: `/v3/users/394780314b1b4baf8a0d7405ed69783f` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:48.489773 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366cb7bd-3bf6-4ac5-9521-e382da16d41c admin admin] RBAC: Authorizing `identity:delete_user(user_id=394780314b1b4baf8a0d7405ed69783f)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:48.491718 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366cb7bd-3bf6-4ac5-9521-e382da16d41c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:48.498267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1050/2097] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:48.502370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0290cbd-7000-4f0a-aa99-8d0345713c8b admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:48.512751 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0290cbd-7000-4f0a-aa99-8d0345713c8b admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:48.518050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d00119-9684-4885-8960-c0d09f9b2e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:48.519017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d00119-9684-4885-8960-c0d09f9b2e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:48.519356 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d00119-9684-4885-8960-c0d09f9b2e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:48.520014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d00119-9684-4885-8960-c0d09f9b2e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:48.520790 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d00119-9684-4885-8960-c0d09f9b2e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:48.522539 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d00119-9684-4885-8960-c0d09f9b2e12 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:48.534580 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1051/2098] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:48.538517 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d00119-9684-4885-8960-c0d09f9b2e12 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:48.543493 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-366cb7bd-3bf6-4ac5-9521-e382da16d41c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '394780314b1b4baf8a0d7405ed69783f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:48.550845 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d00119-9684-4885-8960-c0d09f9b2e12 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:48.557435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e1129-90ee-4c27-ab00-c47a3f70cf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:48.558072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-366cb7bd-3bf6-4ac5-9521-e382da16d41c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '394780314b1b4baf8a0d7405ed69783f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:48.558339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e1129-90ee-4c27-ab00-c47a3f70cf02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:48.558630 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e1129-90ee-4c27-ab00-c47a3f70cf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:48.558937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e1129-90ee-4c27-ab00-c47a3f70cf02 admin admin] PATH_INFO: `/v3/projects/53fb7ad5bed54e718c18ec0568db89b3/users/5ed2846f9b1b48f3bbe4b55eb10c8105/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:48.565821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-366cb7bd-3bf6-4ac5-9521-e382da16d41c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '394780314b1b4baf8a0d7405ed69783f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:48.566558 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-366cb7bd-3bf6-4ac5-9521-e382da16d41c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '394780314b1b4baf8a0d7405ed69783f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:48.579550 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e1129-90ee-4c27-ab00-c47a3f70cf02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53fb7ad5bed54e718c18ec0568db89b3, user_id=5ed2846f9b1b48f3bbe4b55eb10c8105, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:48.583037 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e1129-90ee-4c27-ab00-c47a3f70cf02 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:48.584583 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-366cb7bd-3bf6-4ac5-9521-e382da16d41c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '394780314b1b4baf8a0d7405ed69783f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:48.597590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1048/2099] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:48 2022] DELETE /identity/v3/users/394780314b1b4baf8a0d7405ed69783f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:48.603913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d1746-e49b-4b84-946d-53ef01af0273 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:48.605578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d1746-e49b-4b84-946d-53ef01af0273 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:48.605941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d1746-e49b-4b84-946d-53ef01af0273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:48.610730 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1052/2100] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:48 2022] PUT /identity/v3/projects/53fb7ad5bed54e718c18ec0568db89b3/users/5ed2846f9b1b48f3bbe4b55eb10c8105/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:48.623492 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb37f05-57a7-406d-b03f-361d5f68faf6 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:48.623916 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb37f05-57a7-406d-b03f-361d5f68faf6 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:48.624281 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb37f05-57a7-406d-b03f-361d5f68faf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:48.630523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-6b0d1746-e49b-4b84-946d-53ef01af0273 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:48.631779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6b0d1746-e49b-4b84-946d-53ef01af0273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:48.670307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-2bb37f05-57a7-406d-b03f-361d5f68faf6 None None] MFA Rules not processed for user `5ed2846f9b1b48f3bbe4b55eb10c8105`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:48.672684 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2bb37f05-57a7-406d-b03f-361d5f68faf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:48.674436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:48.735082 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1049/2101] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:48.754965 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d1746-e49b-4b84-946d-53ef01af0273 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:48.756391 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6b0d1746-e49b-4b84-946d-53ef01af0273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:48.763381 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1053/2102] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:48.767747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb37f05-57a7-406d-b03f-361d5f68faf6 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:48.769364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2bb37f05-57a7-406d-b03f-361d5f68faf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:48.785395 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb37f05-57a7-406d-b03f-361d5f68faf6 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:48.791290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c46e1b-79ce-4744-9620-99a490d458bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:48.792267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c46e1b-79ce-4744-9620-99a490d458bc admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:48.792267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c46e1b-79ce-4744-9620-99a490d458bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:48.792549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c46e1b-79ce-4744-9620-99a490d458bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:48.793348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c46e1b-79ce-4744-9620-99a490d458bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:48.795711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c46e1b-79ce-4744-9620-99a490d458bc admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:48.825759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1054/2103] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:48 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:13:48.826035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d1746-e49b-4b84-946d-53ef01af0273 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:48.829596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c46e1b-79ce-4744-9620-99a490d458bc admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:48.868453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:48.889401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddf981-e8e7-4768-b06e-e02f7fdca7a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:48.890068 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddf981-e8e7-4768-b06e-e02f7fdca7a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:48.890358 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddf981-e8e7-4768-b06e-e02f7fdca7a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:48.890645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddf981-e8e7-4768-b06e-e02f7fdca7a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:48.903878 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c46e1b-79ce-4744-9620-99a490d458bc admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:48.909490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c8067-7796-4ec8-9b41-99b25e120776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:48.910240 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c8067-7796-4ec8-9b41-99b25e120776 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:48.910535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c8067-7796-4ec8-9b41-99b25e120776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:48.910926 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c8067-7796-4ec8-9b41-99b25e120776 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:48.911754 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c8067-7796-4ec8-9b41-99b25e120776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:48.913975 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c8067-7796-4ec8-9b41-99b25e120776 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:48.942577 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1055/2104] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:48 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:13:48.943462 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c8067-7796-4ec8-9b41-99b25e120776 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:48.957313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c8067-7796-4ec8-9b41-99b25e120776 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:48.964600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b1aec-bee1-404a-8145-1c0ec05d9e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:48.964600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b1aec-bee1-404a-8145-1c0ec05d9e54 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:48.964600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b1aec-bee1-404a-8145-1c0ec05d9e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:48.964893 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b1aec-bee1-404a-8145-1c0ec05d9e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:48.965429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b1aec-bee1-404a-8145-1c0ec05d9e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:48.966005 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ddf981-e8e7-4768-b06e-e02f7fdca7a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:48.967090 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b1aec-bee1-404a-8145-1c0ec05d9e54 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:48.967659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ddf981-e8e7-4768-b06e-e02f7fdca7a3 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:48.981961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1056/2105] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:48.982198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1050/2106] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:48.983662 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b1aec-bee1-404a-8145-1c0ec05d9e54 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:48.996414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b1aec-bee1-404a-8145-1c0ec05d9e54 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:49.001125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499711b3-ce73-4df0-8e6c-d9c6e8109884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:49.002038 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499711b3-ce73-4df0-8e6c-d9c6e8109884 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:49.002500 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499711b3-ce73-4df0-8e6c-d9c6e8109884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:49.002945 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499711b3-ce73-4df0-8e6c-d9c6e8109884 admin admin] PATH_INFO: `/v3/projects/933c56f53a7744b3a3a4b35b80faffe0/users/9f49265f607f42cd9b7e485a541612bd/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:49.019491 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499711b3-ce73-4df0-8e6c-d9c6e8109884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=933c56f53a7744b3a3a4b35b80faffe0, user_id=9f49265f607f42cd9b7e485a541612bd, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:49.022472 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499711b3-ce73-4df0-8e6c-d9c6e8109884 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:49.037767 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1057/2107] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:48 2022] PUT /identity/v3/projects/933c56f53a7744b3a3a4b35b80faffe0/users/9f49265f607f42cd9b7e485a541612bd/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:49.041995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddf981-e8e7-4768-b06e-e02f7fdca7a3 service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:49.052358 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddf981-e8e7-4768-b06e-e02f7fdca7a3 service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:49.104490 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:49.130268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd636db-b1b3-45ab-bcd5-983d941f1b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:49.131789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd636db-b1b3-45ab-bcd5-983d941f1b69 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:49.132085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd636db-b1b3-45ab-bcd5-983d941f1b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:49.132362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd636db-b1b3-45ab-bcd5-983d941f1b69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:49.133622 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd636db-b1b3-45ab-bcd5-983d941f1b69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:49.135463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd636db-b1b3-45ab-bcd5-983d941f1b69 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:49.150805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1051/2108] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:49.151573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499711b3-ce73-4df0-8e6c-d9c6e8109884 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:49.162721 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499711b3-ce73-4df0-8e6c-d9c6e8109884 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:49.180121 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17541896-6148-4e85-81ca-fa8b58190898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:49.181041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17541896-6148-4e85-81ca-fa8b58190898 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:49.185324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17541896-6148-4e85-81ca-fa8b58190898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:49.185690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17541896-6148-4e85-81ca-fa8b58190898 admin admin] PATH_INFO: `/v3/projects/933c56f53a7744b3a3a4b35b80faffe0/users/9f49265f607f42cd9b7e485a541612bd/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:49.189006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17541896-6148-4e85-81ca-fa8b58190898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=933c56f53a7744b3a3a4b35b80faffe0, user_id=9f49265f607f42cd9b7e485a541612bd, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:49.191806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17541896-6148-4e85-81ca-fa8b58190898 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:49.211252 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1058/2109] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:49 2022] PUT /identity/v3/projects/933c56f53a7744b3a3a4b35b80faffe0/users/9f49265f607f42cd9b7e485a541612bd/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:49.221863 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeab146-a0bc-4abd-b7c3-da7c2ccffcc8 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:49.222321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeab146-a0bc-4abd-b7c3-da7c2ccffcc8 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:49.222435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeab146-a0bc-4abd-b7c3-da7c2ccffcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:49.256211 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-7aeab146-a0bc-4abd-b7c3-da7c2ccffcc8 None None] MFA Rules not processed for user `9f49265f607f42cd9b7e485a541612bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:49.257543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-7aeab146-a0bc-4abd-b7c3-da7c2ccffcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:49.327520 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17541896-6148-4e85-81ca-fa8b58190898 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:49.329440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-17541896-6148-4e85-81ca-fa8b58190898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:49.367053 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1052/2110] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:49 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:49.377497 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeab146-a0bc-4abd-b7c3-da7c2ccffcc8 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:49.379676 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-7aeab146-a0bc-4abd-b7c3-da7c2ccffcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:49.423626 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17541896-6148-4e85-81ca-fa8b58190898 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:49.429099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283751a-457a-4731-98af-9c134281aefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:49.430715 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283751a-457a-4731-98af-9c134281aefd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:49.430990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283751a-457a-4731-98af-9c134281aefd admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:49.431269 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283751a-457a-4731-98af-9c134281aefd admin admin] PATH_INFO: `/v3/projects/cbaca913aca2411cbbade4fd5236c0a1` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:49.432476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283751a-457a-4731-98af-9c134281aefd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbaca913aca2411cbbade4fd5236c0a1)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:49.434379 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283751a-457a-4731-98af-9c134281aefd admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:49.443129 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeab146-a0bc-4abd-b7c3-da7c2ccffcc8 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:49.448032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee210f0-4087-4fb3-98ca-7d254fb07281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:49.448937 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee210f0-4087-4fb3-98ca-7d254fb07281 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:49.449224 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee210f0-4087-4fb3-98ca-7d254fb07281 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:49.449505 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee210f0-4087-4fb3-98ca-7d254fb07281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:49.458725 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee210f0-4087-4fb3-98ca-7d254fb07281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:49.460171 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee210f0-4087-4fb3-98ca-7d254fb07281 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:49.472739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1053/2111] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:49 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:49.483685 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0283751a-457a-4731-98af-9c134281aefd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbaca913aca2411cbbade4fd5236c0a1', 'request_id': 'req-0283751a-457a-4731-98af-9c134281aefd'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:49.489314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0283751a-457a-4731-98af-9c134281aefd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbaca913aca2411cbbade4fd5236c0a1', 'request_id': 'req-0283751a-457a-4731-98af-9c134281aefd'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:49.496216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0283751a-457a-4731-98af-9c134281aefd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbaca913aca2411cbbade4fd5236c0a1', 'request_id': 'req-0283751a-457a-4731-98af-9c134281aefd'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:49.511842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0283751a-457a-4731-98af-9c134281aefd admin admin] The token cache is being invalidate because project cbaca913aca2411cbbade4fd5236c0a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:49.512160 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0283751a-457a-4731-98af-9c134281aefd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:49.512815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0283751a-457a-4731-98af-9c134281aefd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:49.514403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1059/2112] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:49 2022] DELETE /identity/v3/projects/cbaca913aca2411cbbade4fd5236c0a1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:49.518458 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee210f0-4087-4fb3-98ca-7d254fb07281 service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:49.529719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee210f0-4087-4fb3-98ca-7d254fb07281 service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:49.546317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283751a-457a-4731-98af-9c134281aefd admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:49.548245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-0283751a-457a-4731-98af-9c134281aefd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:49.555785 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:49.589850 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a3826-5eee-443d-99e5-8affd20e06a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:49.590801 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a3826-5eee-443d-99e5-8affd20e06a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:49.591109 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a3826-5eee-443d-99e5-8affd20e06a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:49.591388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a3826-5eee-443d-99e5-8affd20e06a5 admin admin] PATH_INFO: `/v3/users/84bbd2eecd474802af33f1f5155590c1` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:49.592609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a3826-5eee-443d-99e5-8affd20e06a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84bbd2eecd474802af33f1f5155590c1)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:49.595003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a3826-5eee-443d-99e5-8affd20e06a5 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:49.600487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:49.611184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283751a-457a-4731-98af-9c134281aefd admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:49.615568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167d1b5-f7cc-4049-bf5f-05ca064eca4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:49.616227 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167d1b5-f7cc-4049-bf5f-05ca064eca4a service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:49.616514 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167d1b5-f7cc-4049-bf5f-05ca064eca4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:49.616797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167d1b5-f7cc-4049-bf5f-05ca064eca4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:49.619276 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7167d1b5-f7cc-4049-bf5f-05ca064eca4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:49.646067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-908a3826-5eee-443d-99e5-8affd20e06a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84bbd2eecd474802af33f1f5155590c1'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:49.659345 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-908a3826-5eee-443d-99e5-8affd20e06a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84bbd2eecd474802af33f1f5155590c1'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:49.667123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-908a3826-5eee-443d-99e5-8affd20e06a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84bbd2eecd474802af33f1f5155590c1'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:49.667848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-908a3826-5eee-443d-99e5-8affd20e06a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84bbd2eecd474802af33f1f5155590c1'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:49.675493 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-908a3826-5eee-443d-99e5-8affd20e06a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84bbd2eecd474802af33f1f5155590c1'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:49.682382 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1054/2113] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:49 2022] DELETE /identity/v3/users/84bbd2eecd474802af33f1f5155590c1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:49.685261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167d1b5-f7cc-4049-bf5f-05ca064eca4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:49.686852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167d1b5-f7cc-4049-bf5f-05ca064eca4a service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:49.696320 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1060/2114] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:50.000623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a3826-5eee-443d-99e5-8affd20e06a5 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:50.002070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-908a3826-5eee-443d-99e5-8affd20e06a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:50.047990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167d1b5-f7cc-4049-bf5f-05ca064eca4a service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:50.049519 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7167d1b5-f7cc-4049-bf5f-05ca064eca4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:50.075269 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a3826-5eee-443d-99e5-8affd20e06a5 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:50.075887 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167d1b5-f7cc-4049-bf5f-05ca064eca4a service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:50.080445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54525dea-9f24-4268-b7ed-adca44f72c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:50.080728 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2444f79c-2865-4dd4-9ec6-bc9b20d64e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:50.081329 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54525dea-9f24-4268-b7ed-adca44f72c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.081606 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54525dea-9f24-4268-b7ed-adca44f72c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.081700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444f79c-2865-4dd4-9ec6-bc9b20d64e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.081871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54525dea-9f24-4268-b7ed-adca44f72c63 admin admin] PATH_INFO: `/v3/users/5ed2846f9b1b48f3bbe4b55eb10c8105` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.082424 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444f79c-2865-4dd4-9ec6-bc9b20d64e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.082764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444f79c-2865-4dd4-9ec6-bc9b20d64e75 admin admin] PATH_INFO: `/v3/projects/5dea97c414ce4acf907e3b525774ead4` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.083221 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54525dea-9f24-4268-b7ed-adca44f72c63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ed2846f9b1b48f3bbe4b55eb10c8105)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:50.083967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2444f79c-2865-4dd4-9ec6-bc9b20d64e75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dea97c414ce4acf907e3b525774ead4)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:50.085257 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54525dea-9f24-4268-b7ed-adca44f72c63 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:50.086641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2444f79c-2865-4dd4-9ec6-bc9b20d64e75 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:50.133545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-54525dea-9f24-4268-b7ed-adca44f72c63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ed2846f9b1b48f3bbe4b55eb10c8105'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:50.135369 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2444f79c-2865-4dd4-9ec6-bc9b20d64e75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dea97c414ce4acf907e3b525774ead4', 'request_id': 'req-2444f79c-2865-4dd4-9ec6-bc9b20d64e75'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:50.140941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2444f79c-2865-4dd4-9ec6-bc9b20d64e75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dea97c414ce4acf907e3b525774ead4', 'request_id': 'req-2444f79c-2865-4dd4-9ec6-bc9b20d64e75'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:50.146007 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-54525dea-9f24-4268-b7ed-adca44f72c63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ed2846f9b1b48f3bbe4b55eb10c8105'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:50.147156 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2444f79c-2865-4dd4-9ec6-bc9b20d64e75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dea97c414ce4acf907e3b525774ead4', 'request_id': 'req-2444f79c-2865-4dd4-9ec6-bc9b20d64e75'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:50.153980 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-54525dea-9f24-4268-b7ed-adca44f72c63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ed2846f9b1b48f3bbe4b55eb10c8105'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:50.154756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-54525dea-9f24-4268-b7ed-adca44f72c63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ed2846f9b1b48f3bbe4b55eb10c8105'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:50.163168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-54525dea-9f24-4268-b7ed-adca44f72c63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ed2846f9b1b48f3bbe4b55eb10c8105'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:50.166190 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2444f79c-2865-4dd4-9ec6-bc9b20d64e75 admin admin] The token cache is being invalidate because project 5dea97c414ce4acf907e3b525774ead4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:50.166488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2444f79c-2865-4dd4-9ec6-bc9b20d64e75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:50.167167 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2444f79c-2865-4dd4-9ec6-bc9b20d64e75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:50.169056 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1061/2115] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:50 2022] DELETE /identity/v3/projects/5dea97c414ce4acf907e3b525774ead4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:50.173394 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1055/2116] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:49 2022] DELETE /identity/v3/users/5ed2846f9b1b48f3bbe4b55eb10c8105 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:50.173546 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e139ea-9ac4-40ea-ba47-aa76a1fcda46 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.173704 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e139ea-9ac4-40ea-ba47-aa76a1fcda46 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.173998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e139ea-9ac4-40ea-ba47-aa76a1fcda46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.179162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304d5d4-1762-4571-90a2-fd02f9def538 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.179520 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304d5d4-1762-4571-90a2-fd02f9def538 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.180051 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304d5d4-1762-4571-90a2-fd02f9def538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.186874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:50.194122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-d6e139ea-9ac4-40ea-ba47-aa76a1fcda46 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:50.195933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d6e139ea-9ac4-40ea-ba47-aa76a1fcda46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:50.202017 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-7304d5d4-1762-4571-90a2-fd02f9def538 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:50.204310 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-7304d5d4-1762-4571-90a2-fd02f9def538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:50.245067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:50.270881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1062/2117] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:50.279331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4d380-8d09-4f30-9f5c-701d30c0c65e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.279626 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4d380-8d09-4f30-9f5c-701d30c0c65e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.279903 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4d380-8d09-4f30-9f5c-701d30c0c65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.296397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1056/2118] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:50.299261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-f7f4d380-8d09-4f30-9f5c-701d30c0c65e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:50.300908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f7f4d380-8d09-4f30-9f5c-701d30c0c65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:50.306825 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304d5d4-1762-4571-90a2-fd02f9def538 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:50.309254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1063/2119] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:50.309982 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-7304d5d4-1762-4571-90a2-fd02f9def538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:50.313396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4d380-8d09-4f30-9f5c-701d30c0c65e None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:50.322903 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4d380-8d09-4f30-9f5c-701d30c0c65e None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:50.327798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea6de8-7032-41ba-b6e6-711085a23716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:50.328447 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea6de8-7032-41ba-b6e6-711085a23716 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.328869 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea6de8-7032-41ba-b6e6-711085a23716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.329151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea6de8-7032-41ba-b6e6-711085a23716 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.329874 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea6de8-7032-41ba-b6e6-711085a23716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:50.330183 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea6de8-7032-41ba-b6e6-711085a23716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:50.331796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea6de8-7032-41ba-b6e6-711085a23716 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:50.344183 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1064/2120] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:13:50.348427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea6de8-7032-41ba-b6e6-711085a23716 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:50.358104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea6de8-7032-41ba-b6e6-711085a23716 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:50.363420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6b0c0-00e4-4387-b192-2594f575d74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:50.364072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6b0c0-00e4-4387-b192-2594f575d74d admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.364356 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6b0c0-00e4-4387-b192-2594f575d74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.364632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b6b0c0-00e4-4387-b192-2594f575d74d admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.365313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6b0c0-00e4-4387-b192-2594f575d74d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:50.369574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b6b0c0-00e4-4387-b192-2594f575d74d admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:50.372815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304d5d4-1762-4571-90a2-fd02f9def538 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:50.376562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2224a-0188-4154-b6d6-b780097548bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:50.377224 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2224a-0188-4154-b6d6-b780097548bc service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.377514 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2224a-0188-4154-b6d6-b780097548bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.377800 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2224a-0188-4154-b6d6-b780097548bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.392110 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2224a-0188-4154-b6d6-b780097548bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:50.393577 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2224a-0188-4154-b6d6-b780097548bc service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:50.398021 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1065/2121] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:50 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:13:50.402461 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6b0c0-00e4-4387-b192-2594f575d74d admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:50.404892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1057/2122] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:13:50.481152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b6b0c0-00e4-4387-b192-2594f575d74d admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:50.485962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cfc4c-292b-47b5-b5d4-5db8c914bd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:50.486775 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cfc4c-292b-47b5-b5d4-5db8c914bd7d admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.487063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cfc4c-292b-47b5-b5d4-5db8c914bd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.487339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cfc4c-292b-47b5-b5d4-5db8c914bd7d admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.488137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cfc4c-292b-47b5-b5d4-5db8c914bd7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:50.490188 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cfc4c-292b-47b5-b5d4-5db8c914bd7d admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:50.514706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1066/2123] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:50 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:13:50.516098 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2224a-0188-4154-b6d6-b780097548bc service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:50.526991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2224a-0188-4154-b6d6-b780097548bc service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:50.531901 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f841f8-956d-44ee-97fa-74bef2ae90ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:50.532569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f841f8-956d-44ee-97fa-74bef2ae90ed admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.532858 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f841f8-956d-44ee-97fa-74bef2ae90ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.533140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f841f8-956d-44ee-97fa-74bef2ae90ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.534386 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f841f8-956d-44ee-97fa-74bef2ae90ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:50.536058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f841f8-956d-44ee-97fa-74bef2ae90ed admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:50.550412 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1058/2124] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:50.555547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cfc4c-292b-47b5-b5d4-5db8c914bd7d admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:50.566218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cfc4c-292b-47b5-b5d4-5db8c914bd7d admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:50.571785 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf358f9-b780-4b9a-a967-6d4706942850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:50.572525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf358f9-b780-4b9a-a967-6d4706942850 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.572849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf358f9-b780-4b9a-a967-6d4706942850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.573406 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf358f9-b780-4b9a-a967-6d4706942850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.574197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf358f9-b780-4b9a-a967-6d4706942850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:50.575855 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf358f9-b780-4b9a-a967-6d4706942850 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:50.586952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1067/2125] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:50.591199 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f841f8-956d-44ee-97fa-74bef2ae90ed admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:50.601683 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f841f8-956d-44ee-97fa-74bef2ae90ed admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:50.606551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee3b49-6d9f-43cf-b5a6-fb5745b6f85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:50.607466 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee3b49-6d9f-43cf-b5a6-fb5745b6f85b admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.607771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee3b49-6d9f-43cf-b5a6-fb5745b6f85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.608037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee3b49-6d9f-43cf-b5a6-fb5745b6f85b admin admin] PATH_INFO: `/v3/projects/a1ecb2ef4bdd405b942ca70ffdaba8ea/users/3e6e86c1ff9d430bac353aa8bd4f3418/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.624845 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee3b49-6d9f-43cf-b5a6-fb5745b6f85b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1ecb2ef4bdd405b942ca70ffdaba8ea, user_id=3e6e86c1ff9d430bac353aa8bd4f3418, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:50.627550 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee3b49-6d9f-43cf-b5a6-fb5745b6f85b admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:50.641872 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1059/2126] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:50 2022] PUT /identity/v3/projects/a1ecb2ef4bdd405b942ca70ffdaba8ea/users/3e6e86c1ff9d430bac353aa8bd4f3418/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:50.652406 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c63bcd-bfdd-4992-b592-b16ba27bd4d6 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.652763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c63bcd-bfdd-4992-b592-b16ba27bd4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.653084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c63bcd-bfdd-4992-b592-b16ba27bd4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.683020 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-82c63bcd-bfdd-4992-b592-b16ba27bd4d6 None None] MFA Rules not processed for user `3e6e86c1ff9d430bac353aa8bd4f3418`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:50.684260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-82c63bcd-bfdd-4992-b592-b16ba27bd4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:50.686928 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee3b49-6d9f-43cf-b5a6-fb5745b6f85b admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:50.688621 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-47ee3b49-6d9f-43cf-b5a6-fb5745b6f85b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:50.761240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee3b49-6d9f-43cf-b5a6-fb5745b6f85b admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:50.765946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:50.766200 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1068/2127] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:50.770269 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.770566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.770814 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c63bcd-bfdd-4992-b592-b16ba27bd4d6 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:50.770914 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8 admin admin] PATH_INFO: `/v3/projects/53fb7ad5bed54e718c18ec0568db89b3` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.772202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53fb7ad5bed54e718c18ec0568db89b3)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:50.774204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:50.781481 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c63bcd-bfdd-4992-b592-b16ba27bd4d6 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:50.786049 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43649f-1f84-46df-92e0-44a9a8a20a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:50.786511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:50.786727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43649f-1f84-46df-92e0-44a9a8a20a50 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.787012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43649f-1f84-46df-92e0-44a9a8a20a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.787296 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43649f-1f84-46df-92e0-44a9a8a20a50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.788018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43649f-1f84-46df-92e0-44a9a8a20a50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:50.790638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43649f-1f84-46df-92e0-44a9a8a20a50 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:50.794987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:50.818711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1069/2128] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:50 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:13:50.822063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43649f-1f84-46df-92e0-44a9a8a20a50 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:50.828755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53fb7ad5bed54e718c18ec0568db89b3', 'request_id': 'req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:50.834324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53fb7ad5bed54e718c18ec0568db89b3', 'request_id': 'req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:50.842776 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53fb7ad5bed54e718c18ec0568db89b3', 'request_id': 'req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:50.857557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8 admin admin] The token cache is being invalidate because project 53fb7ad5bed54e718c18ec0568db89b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:50.857851 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:50.858523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:50.860183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1060/2129] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:50 2022] DELETE /identity/v3/projects/53fb7ad5bed54e718c18ec0568db89b3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:50.864107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:50.865751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:50.884080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43649f-1f84-46df-92e0-44a9a8a20a50 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:50.888730 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb2f5e-c670-4b0d-afd3-7df207324309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:50.889403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb2f5e-c670-4b0d-afd3-7df207324309 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.889685 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb2f5e-c670-4b0d-afd3-7df207324309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.889961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb2f5e-c670-4b0d-afd3-7df207324309 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.890664 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb2f5e-c670-4b0d-afd3-7df207324309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:50.892666 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb2f5e-c670-4b0d-afd3-7df207324309 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:50.915331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1070/2130] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:50 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:50.919231 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb2f5e-c670-4b0d-afd3-7df207324309 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:50.920904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-3ebb2f5e-c670-4b0d-afd3-7df207324309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:50.934017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb2f5e-c670-4b0d-afd3-7df207324309 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:50.938476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d9487-a8d6-431a-a2b0-ff74c45cc3a8 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:50.939236 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1972a7-ccd9-4b94-ad29-c1024c8ec502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:50.939964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1972a7-ccd9-4b94-ad29-c1024c8ec502 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.940189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1972a7-ccd9-4b94-ad29-c1024c8ec502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.940472 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1972a7-ccd9-4b94-ad29-c1024c8ec502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.941200 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1972a7-ccd9-4b94-ad29-c1024c8ec502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:50.942809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1972a7-ccd9-4b94-ad29-c1024c8ec502 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:50.943556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a000af4-a234-4ab0-b960-4274176f63e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:50.944458 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a000af4-a234-4ab0-b960-4274176f63e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.944745 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a000af4-a234-4ab0-b960-4274176f63e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.945298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a000af4-a234-4ab0-b960-4274176f63e4 admin admin] PATH_INFO: `/v3/users/9f49265f607f42cd9b7e485a541612bd` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.946548 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a000af4-a234-4ab0-b960-4274176f63e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f49265f607f42cd9b7e485a541612bd)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:50.948901 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a000af4-a234-4ab0-b960-4274176f63e4 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:50.953123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1071/2131] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:50.957078 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1972a7-ccd9-4b94-ad29-c1024c8ec502 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:50.967008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1972a7-ccd9-4b94-ad29-c1024c8ec502 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:50.971882 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895de652-2392-4fbb-9e85-1ab0e6c3dbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:50.972526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895de652-2392-4fbb-9e85-1ab0e6c3dbd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:50.972884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895de652-2392-4fbb-9e85-1ab0e6c3dbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:50.973244 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895de652-2392-4fbb-9e85-1ab0e6c3dbd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:50.974016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895de652-2392-4fbb-9e85-1ab0e6c3dbd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:50.975880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895de652-2392-4fbb-9e85-1ab0e6c3dbd3 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:50.989070 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1072/2132] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:50.994010 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895de652-2392-4fbb-9e85-1ab0e6c3dbd3 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:50.995096 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6a000af4-a234-4ab0-b960-4274176f63e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f49265f607f42cd9b7e485a541612bd'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:51.005099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895de652-2392-4fbb-9e85-1ab0e6c3dbd3 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:51.009044 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6a000af4-a234-4ab0-b960-4274176f63e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f49265f607f42cd9b7e485a541612bd'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:51.010458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ada46-6891-4e4a-945e-acf70c9a0da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:51.011378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ada46-6891-4e4a-945e-acf70c9a0da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.011659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ada46-6891-4e4a-945e-acf70c9a0da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.011983 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ada46-6891-4e4a-945e-acf70c9a0da2 admin admin] PATH_INFO: `/v3/projects/37948a13953a42a9bc0c0c6eb3bc420a/users/a5a909f4c7614b4dbdc567208d0c3536/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.019388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6a000af4-a234-4ab0-b960-4274176f63e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f49265f607f42cd9b7e485a541612bd'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:51.020107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6a000af4-a234-4ab0-b960-4274176f63e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f49265f607f42cd9b7e485a541612bd'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:51.028256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ada46-6891-4e4a-945e-acf70c9a0da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37948a13953a42a9bc0c0c6eb3bc420a, user_id=a5a909f4c7614b4dbdc567208d0c3536, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:51.030906 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6a000af4-a234-4ab0-b960-4274176f63e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f49265f607f42cd9b7e485a541612bd'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:51.031100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ada46-6891-4e4a-945e-acf70c9a0da2 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:51.039198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1061/2133] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:50 2022] DELETE /identity/v3/users/9f49265f607f42cd9b7e485a541612bd => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:51.046210 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1073/2134] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:50 2022] PUT /identity/v3/projects/37948a13953a42a9bc0c0c6eb3bc420a/users/a5a909f4c7614b4dbdc567208d0c3536/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:51.057263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b1413-ef60-4389-a25a-817ac3a3c599 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.057720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b1413-ef60-4389-a25a-817ac3a3c599 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.057829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b1413-ef60-4389-a25a-817ac3a3c599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.088852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-0a6b1413-ef60-4389-a25a-817ac3a3c599 None None] MFA Rules not processed for user `a5a909f4c7614b4dbdc567208d0c3536`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:51.090114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0a6b1413-ef60-4389-a25a-817ac3a3c599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:51.104633 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:51.181811 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1062/2135] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:51.188181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ada46-6891-4e4a-945e-acf70c9a0da2 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:51.189920 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-918ada46-6891-4e4a-945e-acf70c9a0da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:51.271598 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ada46-6891-4e4a-945e-acf70c9a0da2 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:51.277770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d980bfa-5c4c-4462-abaa-6cb418358ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:51.278257 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d980bfa-5c4c-4462-abaa-6cb418358ecd admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.278634 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d980bfa-5c4c-4462-abaa-6cb418358ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.279009 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d980bfa-5c4c-4462-abaa-6cb418358ecd admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.279883 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d980bfa-5c4c-4462-abaa-6cb418358ecd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:51.281698 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d980bfa-5c4c-4462-abaa-6cb418358ecd admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:51.317426 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1074/2136] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:51 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:51.318652 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b1413-ef60-4389-a25a-817ac3a3c599 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:51.332565 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b1413-ef60-4389-a25a-817ac3a3c599 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:51.333156 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d980bfa-5c4c-4462-abaa-6cb418358ecd admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:51.334906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7d980bfa-5c4c-4462-abaa-6cb418358ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:51.359443 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:51.400908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f3f26-c28e-4fcc-bc3f-3519af2ced34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:51.401702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f3f26-c28e-4fcc-bc3f-3519af2ced34 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.401991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f3f26-c28e-4fcc-bc3f-3519af2ced34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.402275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f3f26-c28e-4fcc-bc3f-3519af2ced34 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.403023 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8f3f26-c28e-4fcc-bc3f-3519af2ced34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:51.404952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8f3f26-c28e-4fcc-bc3f-3519af2ced34 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:51.409870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d980bfa-5c4c-4462-abaa-6cb418358ecd admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:51.415567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7a729-bf2a-4d2c-9a05-a90ef453c60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:51.416510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7a729-bf2a-4d2c-9a05-a90ef453c60e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.416800 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7a729-bf2a-4d2c-9a05-a90ef453c60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.417112 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7a729-bf2a-4d2c-9a05-a90ef453c60e admin admin] PATH_INFO: `/v3/projects/933c56f53a7744b3a3a4b35b80faffe0` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.418470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7a729-bf2a-4d2c-9a05-a90ef453c60e admin admin] RBAC: Authorizing `identity:delete_project(project_id=933c56f53a7744b3a3a4b35b80faffe0)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:51.420564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7a729-bf2a-4d2c-9a05-a90ef453c60e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:51.426719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1063/2137] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:51 2022] POST /identity/v3/users => generated 298 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:51.431189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f3f26-c28e-4fcc-bc3f-3519af2ced34 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:51.443033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f3f26-c28e-4fcc-bc3f-3519af2ced34 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:51.447872 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc8d98-19c5-496f-816b-a8421181bbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:51.449182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc8d98-19c5-496f-816b-a8421181bbda admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.449182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc8d98-19c5-496f-816b-a8421181bbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.449562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc8d98-19c5-496f-816b-a8421181bbda admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.450462 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc8d98-19c5-496f-816b-a8421181bbda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:51.452271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc8d98-19c5-496f-816b-a8421181bbda admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:51.464852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1064/2138] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:51.469502 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc8d98-19c5-496f-816b-a8421181bbda admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:51.482213 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc8d98-19c5-496f-816b-a8421181bbda admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:51.484065 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-67d7a729-bf2a-4d2c-9a05-a90ef453c60e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '933c56f53a7744b3a3a4b35b80faffe0', 'request_id': 'req-67d7a729-bf2a-4d2c-9a05-a90ef453c60e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:51.495612 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-67d7a729-bf2a-4d2c-9a05-a90ef453c60e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '933c56f53a7744b3a3a4b35b80faffe0', 'request_id': 'req-67d7a729-bf2a-4d2c-9a05-a90ef453c60e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:51.503296 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-67d7a729-bf2a-4d2c-9a05-a90ef453c60e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '933c56f53a7744b3a3a4b35b80faffe0', 'request_id': 'req-67d7a729-bf2a-4d2c-9a05-a90ef453c60e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:51.516720 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-67d7a729-bf2a-4d2c-9a05-a90ef453c60e admin admin] The token cache is being invalidate because project 933c56f53a7744b3a3a4b35b80faffe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:51.517008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-67d7a729-bf2a-4d2c-9a05-a90ef453c60e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:51.517647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-67d7a729-bf2a-4d2c-9a05-a90ef453c60e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:51.520166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1075/2139] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:51 2022] DELETE /identity/v3/projects/933c56f53a7744b3a3a4b35b80faffe0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:51.524750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e2618-9645-4c74-ac4b-3f4471a2662f None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.525045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e2618-9645-4c74-ac4b-3f4471a2662f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.525319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e2618-9645-4c74-ac4b-3f4471a2662f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.544985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa2be3-8fb2-4753-9dc1-3196c3ad9f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:51.545465 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-d60e2618-9645-4c74-ac4b-3f4471a2662f None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:51.545932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa2be3-8fb2-4753-9dc1-3196c3ad9f6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.546215 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa2be3-8fb2-4753-9dc1-3196c3ad9f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.546492 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa2be3-8fb2-4753-9dc1-3196c3ad9f6b admin admin] PATH_INFO: `/v3/projects/be918a9e63cb43928a6ee4206890f323/users/9e0f54e2acbb4906b90357e01aae9a5c/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.546696 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d60e2618-9645-4c74-ac4b-3f4471a2662f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:51.555338 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1076/2140] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:51.562966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca549677-6a28-483b-b510-efc4ebc53b5f None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.563257 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca549677-6a28-483b-b510-efc4ebc53b5f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.563528 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca549677-6a28-483b-b510-efc4ebc53b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.564000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa2be3-8fb2-4753-9dc1-3196c3ad9f6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=be918a9e63cb43928a6ee4206890f323, user_id=9e0f54e2acbb4906b90357e01aae9a5c, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:51.566803 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aa2be3-8fb2-4753-9dc1-3196c3ad9f6b admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:51.581054 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1065/2141] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:51 2022] PUT /identity/v3/projects/be918a9e63cb43928a6ee4206890f323/users/9e0f54e2acbb4906b90357e01aae9a5c/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:51.581521 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-ca549677-6a28-483b-b510-efc4ebc53b5f None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:51.582714 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ca549677-6a28-483b-b510-efc4ebc53b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:51.585037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa2be3-8fb2-4753-9dc1-3196c3ad9f6b admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:51.587033 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e3aa2be3-8fb2-4753-9dc1-3196c3ad9f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:51.599634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:51.644155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1077/2142] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:51.648298 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca549677-6a28-483b-b510-efc4ebc53b5f None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:51.654660 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aa2be3-8fb2-4753-9dc1-3196c3ad9f6b admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:51.657115 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca549677-6a28-483b-b510-efc4ebc53b5f None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:51.659617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c116f-ef24-4bab-95e2-6f4be5be24d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:51.660293 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c116f-ef24-4bab-95e2-6f4be5be24d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.660579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c116f-ef24-4bab-95e2-6f4be5be24d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.660870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c116f-ef24-4bab-95e2-6f4be5be24d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.661609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c116f-ef24-4bab-95e2-6f4be5be24d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:51.663263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c116f-ef24-4bab-95e2-6f4be5be24d1 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:51.663542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ec83d-1569-496a-9a31-e6878c6ec7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:51.665763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ec83d-1569-496a-9a31-e6878c6ec7f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.666055 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ec83d-1569-496a-9a31-e6878c6ec7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.666332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ec83d-1569-496a-9a31-e6878c6ec7f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.667111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ec83d-1569-496a-9a31-e6878c6ec7f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:51.667811 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ec83d-1569-496a-9a31-e6878c6ec7f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:51.668993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ec83d-1569-496a-9a31-e6878c6ec7f1 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:51.674306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1066/2143] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:51.678382 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c116f-ef24-4bab-95e2-6f4be5be24d1 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:51.684067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1078/2144] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:13:51.684969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ec83d-1569-496a-9a31-e6878c6ec7f1 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:51.689362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c116f-ef24-4bab-95e2-6f4be5be24d1 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:51.695749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3bb81-3441-4526-b355-e36a5bc72db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:51.696462 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ec83d-1569-496a-9a31-e6878c6ec7f1 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:51.696974 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3bb81-3441-4526-b355-e36a5bc72db6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.697266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3bb81-3441-4526-b355-e36a5bc72db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.697548 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3bb81-3441-4526-b355-e36a5bc72db6 admin admin] PATH_INFO: `/v3/projects/be918a9e63cb43928a6ee4206890f323/users/9e0f54e2acbb4906b90357e01aae9a5c/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.699850 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3bb81-3441-4526-b355-e36a5bc72db6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be918a9e63cb43928a6ee4206890f323, user_id=9e0f54e2acbb4906b90357e01aae9a5c, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:51.701312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4cdf58-5d6c-4257-8d57-b27dfda31af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:51.701945 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cdf58-5d6c-4257-8d57-b27dfda31af9 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.702239 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cdf58-5d6c-4257-8d57-b27dfda31af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.702506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4cdf58-5d6c-4257-8d57-b27dfda31af9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.702598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3bb81-3441-4526-b355-e36a5bc72db6 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:51.703221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4cdf58-5d6c-4257-8d57-b27dfda31af9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:51.705082 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4cdf58-5d6c-4257-8d57-b27dfda31af9 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:51.714717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1067/2145] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:51 2022] PUT /identity/v3/projects/be918a9e63cb43928a6ee4206890f323/users/9e0f54e2acbb4906b90357e01aae9a5c/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:51.725426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174926af-2c28-4b2b-944b-bf50a932736e None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.725844 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174926af-2c28-4b2b-944b-bf50a932736e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.726451 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174926af-2c28-4b2b-944b-bf50a932736e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.736974 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1079/2146] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:51 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:13:51.741070 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4cdf58-5d6c-4257-8d57-b27dfda31af9 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:51.747184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:51.758557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-174926af-2c28-4b2b-944b-bf50a932736e None None] MFA Rules not processed for user `9e0f54e2acbb4906b90357e01aae9a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:51.760125 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-174926af-2c28-4b2b-944b-bf50a932736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:51.822856 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4cdf58-5d6c-4257-8d57-b27dfda31af9 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:51.828744 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c9b63-913f-4abe-88fc-09a31289eaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:51.829457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c9b63-913f-4abe-88fc-09a31289eaaf admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.829734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c9b63-913f-4abe-88fc-09a31289eaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.830066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c9b63-913f-4abe-88fc-09a31289eaaf admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.830852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c9b63-913f-4abe-88fc-09a31289eaaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:51.833002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c9b63-913f-4abe-88fc-09a31289eaaf admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:51.851018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1068/2147] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:51.862894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1080/2148] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:51 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:13:51.863849 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174926af-2c28-4b2b-944b-bf50a932736e None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:51.863965 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c9b63-913f-4abe-88fc-09a31289eaaf admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:51.866137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8d8c9b63-913f-4abe-88fc-09a31289eaaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:51.875625 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174926af-2c28-4b2b-944b-bf50a932736e None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:51.880177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5406f-b37e-4137-ac74-667388df894e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:51.880842 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5406f-b37e-4137-ac74-667388df894e admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.881146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5406f-b37e-4137-ac74-667388df894e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.881407 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5406f-b37e-4137-ac74-667388df894e admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.882132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5406f-b37e-4137-ac74-667388df894e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:51.884911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5406f-b37e-4137-ac74-667388df894e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:51.895068 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1069/2149] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:51.899447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5406f-b37e-4137-ac74-667388df894e admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:51.909162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5406f-b37e-4137-ac74-667388df894e admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:51.913842 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa8ac8-9d74-4121-9da5-217f31b995ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:51.914788 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa8ac8-9d74-4121-9da5-217f31b995ef admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.915076 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa8ac8-9d74-4121-9da5-217f31b995ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.915360 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa8ac8-9d74-4121-9da5-217f31b995ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.916097 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa8ac8-9d74-4121-9da5-217f31b995ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:51.917678 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa8ac8-9d74-4121-9da5-217f31b995ef admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:51.927221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c9b63-913f-4abe-88fc-09a31289eaaf admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:51.928767 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1070/2150] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:51.931748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234509d-feda-49c3-8f84-ad98f7a5d6fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:51.932663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234509d-feda-49c3-8f84-ad98f7a5d6fc service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.932955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234509d-feda-49c3-8f84-ad98f7a5d6fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.933258 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234509d-feda-49c3-8f84-ad98f7a5d6fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.933357 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa8ac8-9d74-4121-9da5-217f31b995ef admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:51.935380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d234509d-feda-49c3-8f84-ad98f7a5d6fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:51.943989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa8ac8-9d74-4121-9da5-217f31b995ef admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:51.950404 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a75461-e52b-4bff-9521-20cff3ceedbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:51.951315 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a75461-e52b-4bff-9521-20cff3ceedbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:51.951600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a75461-e52b-4bff-9521-20cff3ceedbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:51.951876 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a75461-e52b-4bff-9521-20cff3ceedbc admin admin] PATH_INFO: `/v3/projects/048a306747474af98da675803fb6d44c/users/0d3c41b927a34022b85d23be6fcbbd9a/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:51.967030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a75461-e52b-4bff-9521-20cff3ceedbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=048a306747474af98da675803fb6d44c, user_id=0d3c41b927a34022b85d23be6fcbbd9a, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:51.969894 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a75461-e52b-4bff-9521-20cff3ceedbc admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:51.983969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1071/2151] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:51 2022] PUT /identity/v3/projects/048a306747474af98da675803fb6d44c/users/0d3c41b927a34022b85d23be6fcbbd9a/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:52.009763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234509d-feda-49c3-8f84-ad98f7a5d6fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:52.011267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234509d-feda-49c3-8f84-ad98f7a5d6fc service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:52.021861 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1081/2152] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:52.059611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76040c12-9dc7-438c-88c1-90feedfc110b None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:52.060097 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76040c12-9dc7-438c-88c1-90feedfc110b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:52.060241 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76040c12-9dc7-438c-88c1-90feedfc110b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:52.086874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-76040c12-9dc7-438c-88c1-90feedfc110b None None] MFA Rules not processed for user `0d3c41b927a34022b85d23be6fcbbd9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:52.088000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-76040c12-9dc7-438c-88c1-90feedfc110b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:52.114163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:52.169527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1072/2153] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:52.180664 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234509d-feda-49c3-8f84-ad98f7a5d6fc service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:52.182391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d234509d-feda-49c3-8f84-ad98f7a5d6fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:52.196280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234509d-feda-49c3-8f84-ad98f7a5d6fc service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:52.248137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:52.255866 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ec7b9-b518-4899-9be6-b4796ab0c18d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:52.256618 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ec7b9-b518-4899-9be6-b4796ab0c18d service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:52.256958 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ec7b9-b518-4899-9be6-b4796ab0c18d service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:52.257215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ec7b9-b518-4899-9be6-b4796ab0c18d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:52.275657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ec7b9-b518-4899-9be6-b4796ab0c18d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:52.277296 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ec7b9-b518-4899-9be6-b4796ab0c18d service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:52.290671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1082/2154] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:52.513682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76040c12-9dc7-438c-88c1-90feedfc110b None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:52.516780 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-76040c12-9dc7-438c-88c1-90feedfc110b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:52.654263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76040c12-9dc7-438c-88c1-90feedfc110b None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:52.659529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955785cf-bac0-46a8-a919-ac51bb364fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:52.660912 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955785cf-bac0-46a8-a919-ac51bb364fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:52.661409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955785cf-bac0-46a8-a919-ac51bb364fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:52.661822 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955785cf-bac0-46a8-a919-ac51bb364fb9 admin admin] PATH_INFO: `/v3/users/1ca9be4fcbaa472396d9b51206a73321` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:52.663329 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955785cf-bac0-46a8-a919-ac51bb364fb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ca9be4fcbaa472396d9b51206a73321)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:52.666253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955785cf-bac0-46a8-a919-ac51bb364fb9 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:52.731746 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-955785cf-bac0-46a8-a919-ac51bb364fb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ca9be4fcbaa472396d9b51206a73321'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:52.745474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-955785cf-bac0-46a8-a919-ac51bb364fb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ca9be4fcbaa472396d9b51206a73321'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:52.752823 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-955785cf-bac0-46a8-a919-ac51bb364fb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ca9be4fcbaa472396d9b51206a73321'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:52.755230 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-955785cf-bac0-46a8-a919-ac51bb364fb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ca9be4fcbaa472396d9b51206a73321'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:52.763508 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-955785cf-bac0-46a8-a919-ac51bb364fb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ca9be4fcbaa472396d9b51206a73321'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:52.769983 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1073/2155] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:52 2022] DELETE /identity/v3/users/1ca9be4fcbaa472396d9b51206a73321 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:52.775050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c71b7d-5446-4162-b627-d3913c1b82a5 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:52.775342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c71b7d-5446-4162-b627-d3913c1b82a5 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:52.775628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c71b7d-5446-4162-b627-d3913c1b82a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:52.792937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-82c71b7d-5446-4162-b627-d3913c1b82a5 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:52.794151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-82c71b7d-5446-4162-b627-d3913c1b82a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:52.856304 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1083/2156] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:52.865848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955785cf-bac0-46a8-a919-ac51bb364fb9 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:52.868185 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-955785cf-bac0-46a8-a919-ac51bb364fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:52.917808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:52.931374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955785cf-bac0-46a8-a919-ac51bb364fb9 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:52.935139 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0abb01-99ce-4666-a347-85c6c0412a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:52.935783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0abb01-99ce-4666-a347-85c6c0412a7c service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:52.936063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0abb01-99ce-4666-a347-85c6c0412a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:52.936335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0abb01-99ce-4666-a347-85c6c0412a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:52.946046 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0abb01-99ce-4666-a347-85c6c0412a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:52.947971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0abb01-99ce-4666-a347-85c6c0412a7c service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:52.958519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1074/2157] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:53.254720 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c71b7d-5446-4162-b627-d3913c1b82a5 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:53.256205 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-82c71b7d-5446-4162-b627-d3913c1b82a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:53.270313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c71b7d-5446-4162-b627-d3913c1b82a5 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:53.276215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2ecd7-97f3-417a-b733-60f9b186cbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:53.277646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2ecd7-97f3-417a-b733-60f9b186cbe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:53.277888 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2ecd7-97f3-417a-b733-60f9b186cbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:53.278171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2ecd7-97f3-417a-b733-60f9b186cbe8 admin admin] PATH_INFO: `/v3/projects/3d9aa408366d452fac34f7bd1d17f79e` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:53.279483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2ecd7-97f3-417a-b733-60f9b186cbe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d9aa408366d452fac34f7bd1d17f79e)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:53.281317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2ecd7-97f3-417a-b733-60f9b186cbe8 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:53.331717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-40a2ecd7-97f3-417a-b733-60f9b186cbe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d9aa408366d452fac34f7bd1d17f79e', 'request_id': 'req-40a2ecd7-97f3-417a-b733-60f9b186cbe8'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:53.338518 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-40a2ecd7-97f3-417a-b733-60f9b186cbe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d9aa408366d452fac34f7bd1d17f79e', 'request_id': 'req-40a2ecd7-97f3-417a-b733-60f9b186cbe8'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:53.344995 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-40a2ecd7-97f3-417a-b733-60f9b186cbe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d9aa408366d452fac34f7bd1d17f79e', 'request_id': 'req-40a2ecd7-97f3-417a-b733-60f9b186cbe8'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:53.359049 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-40a2ecd7-97f3-417a-b733-60f9b186cbe8 admin admin] The token cache is being invalidate because project 3d9aa408366d452fac34f7bd1d17f79e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:53.359259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-40a2ecd7-97f3-417a-b733-60f9b186cbe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:53.360240 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-40a2ecd7-97f3-417a-b733-60f9b186cbe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:53.362549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1084/2158] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:53 2022] DELETE /identity/v3/projects/3d9aa408366d452fac34f7bd1d17f79e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:53.368641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237bf73-0de4-42d3-98a2-b31e24290282 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:53.369132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237bf73-0de4-42d3-98a2-b31e24290282 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:53.369273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237bf73-0de4-42d3-98a2-b31e24290282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:53.387271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-e237bf73-0de4-42d3-98a2-b31e24290282 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:53.388568 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e237bf73-0de4-42d3-98a2-b31e24290282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:53.462755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1075/2159] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:53.480232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe1c5d-62e7-468e-a79b-e380e9e444da None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:53.480232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe1c5d-62e7-468e-a79b-e380e9e444da None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:53.480520 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe1c5d-62e7-468e-a79b-e380e9e444da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:53.505308 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-7ffe1c5d-62e7-468e-a79b-e380e9e444da None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:53.506799 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7ffe1c5d-62e7-468e-a79b-e380e9e444da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:53.517395 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1085/2160] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:53.521310 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237bf73-0de4-42d3-98a2-b31e24290282 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:53.538749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237bf73-0de4-42d3-98a2-b31e24290282 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:53.545883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9bd0dd-5c26-4f7f-8482-26f8d6b03fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:53.555090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9bd0dd-5c26-4f7f-8482-26f8d6b03fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:53.555459 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9bd0dd-5c26-4f7f-8482-26f8d6b03fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:53.556084 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9bd0dd-5c26-4f7f-8482-26f8d6b03fc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:53.556531 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9bd0dd-5c26-4f7f-8482-26f8d6b03fc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:53.556849 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9bd0dd-5c26-4f7f-8482-26f8d6b03fc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:53.558766 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9bd0dd-5c26-4f7f-8482-26f8d6b03fc7 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:53.583180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1076/2161] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:13:53.587937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe1c5d-62e7-468e-a79b-e380e9e444da None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:53.597574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe1c5d-62e7-468e-a79b-e380e9e444da None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:53.603907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699ddbd-8d46-4117-8efd-ad0d32beb275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:53.604583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699ddbd-8d46-4117-8efd-ad0d32beb275 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:53.604872 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699ddbd-8d46-4117-8efd-ad0d32beb275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:53.605163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699ddbd-8d46-4117-8efd-ad0d32beb275 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:53.605999 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699ddbd-8d46-4117-8efd-ad0d32beb275 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:53.608089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699ddbd-8d46-4117-8efd-ad0d32beb275 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:53.649833 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1086/2162] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:53 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:53.653650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9bd0dd-5c26-4f7f-8482-26f8d6b03fc7 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:53.667560 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9bd0dd-5c26-4f7f-8482-26f8d6b03fc7 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:53.715506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699ddbd-8d46-4117-8efd-ad0d32beb275 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:53.717616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b699ddbd-8d46-4117-8efd-ad0d32beb275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:53.756934 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbc41b-8183-4b56-8d97-5d60fda34192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:53.758227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbc41b-8183-4b56-8d97-5d60fda34192 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:53.758445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbc41b-8183-4b56-8d97-5d60fda34192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:53.758897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbc41b-8183-4b56-8d97-5d60fda34192 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:53.759916 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbc41b-8183-4b56-8d97-5d60fda34192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:53.761794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbc41b-8183-4b56-8d97-5d60fda34192 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:53.791999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1077/2163] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:53 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:53.801755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbc41b-8183-4b56-8d97-5d60fda34192 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:53.807384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:53.818424 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699ddbd-8d46-4117-8efd-ad0d32beb275 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:53.820969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbc41b-8183-4b56-8d97-5d60fda34192 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:53.826392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d583b4-d4ea-464d-aedf-fd9f262c973a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:53.828345 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d583b4-d4ea-464d-aedf-fd9f262c973a service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:53.828434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15dca3-23ed-432a-8256-9dfbe19aaff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:53.828548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d583b4-d4ea-464d-aedf-fd9f262c973a service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:53.828548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d583b4-d4ea-464d-aedf-fd9f262c973a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:53.829284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15dca3-23ed-432a-8256-9dfbe19aaff1 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:53.829686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15dca3-23ed-432a-8256-9dfbe19aaff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:53.830021 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15dca3-23ed-432a-8256-9dfbe19aaff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:53.830658 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-85d583b4-d4ea-464d-aedf-fd9f262c973a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:53.831219 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15dca3-23ed-432a-8256-9dfbe19aaff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:53.833673 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15dca3-23ed-432a-8256-9dfbe19aaff1 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:53.847271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1078/2164] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:53.851808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15dca3-23ed-432a-8256-9dfbe19aaff1 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:53.862776 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15dca3-23ed-432a-8256-9dfbe19aaff1 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:53.868301 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ea65c-06fb-4f25-8bea-4aa57c8af453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:53.869160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ea65c-06fb-4f25-8bea-4aa57c8af453 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:53.869375 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ea65c-06fb-4f25-8bea-4aa57c8af453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:53.869722 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ea65c-06fb-4f25-8bea-4aa57c8af453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:53.870429 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ea65c-06fb-4f25-8bea-4aa57c8af453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:53.872346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ea65c-06fb-4f25-8bea-4aa57c8af453 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:53.884828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1079/2165] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:53.889353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ea65c-06fb-4f25-8bea-4aa57c8af453 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:53.902283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ea65c-06fb-4f25-8bea-4aa57c8af453 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:53.910717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d583b4-d4ea-464d-aedf-fd9f262c973a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:53.918087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d583b4-d4ea-464d-aedf-fd9f262c973a service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:53.918177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d534c43-eac3-41ff-9912-d5fba3a5a747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:53.918177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d534c43-eac3-41ff-9912-d5fba3a5a747 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:53.918177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d534c43-eac3-41ff-9912-d5fba3a5a747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:53.918177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d534c43-eac3-41ff-9912-d5fba3a5a747 admin admin] PATH_INFO: `/v3/projects/19570f5948f2444596e88411c6996ea1/users/0ca2e554196e46c4ad9100622fed470c/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:53.925900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1087/2166] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:53.946807 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d534c43-eac3-41ff-9912-d5fba3a5a747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19570f5948f2444596e88411c6996ea1, user_id=0ca2e554196e46c4ad9100622fed470c, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:53.950667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d534c43-eac3-41ff-9912-d5fba3a5a747 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:53.968715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1080/2167] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:53 2022] PUT /identity/v3/projects/19570f5948f2444596e88411c6996ea1/users/0ca2e554196e46c4ad9100622fed470c/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:53.980473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3fc88-eb88-4550-b380-d7334c1b0e08 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:53.980784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3fc88-eb88-4550-b380-d7334c1b0e08 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:53.981048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3fc88-eb88-4550-b380-d7334c1b0e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:54.014969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-40d3fc88-eb88-4550-b380-d7334c1b0e08 None None] MFA Rules not processed for user `0ca2e554196e46c4ad9100622fed470c`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:54.016726 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-40d3fc88-eb88-4550-b380-d7334c1b0e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:54.095639 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1088/2168] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:54.100183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d534c43-eac3-41ff-9912-d5fba3a5a747 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:54.120630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:54.181785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d534c43-eac3-41ff-9912-d5fba3a5a747 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:54.187414 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715043f-fac5-4ad5-bcdd-e43c5c743f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:54.188163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715043f-fac5-4ad5-bcdd-e43c5c743f2e admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:54.188451 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715043f-fac5-4ad5-bcdd-e43c5c743f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:54.188732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715043f-fac5-4ad5-bcdd-e43c5c743f2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:54.189446 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f715043f-fac5-4ad5-bcdd-e43c5c743f2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:54.191625 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f715043f-fac5-4ad5-bcdd-e43c5c743f2e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:54.219464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1081/2169] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:54 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:13:54.223267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3fc88-eb88-4550-b380-d7334c1b0e08 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:54.236452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3fc88-eb88-4550-b380-d7334c1b0e08 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:54.299225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86488b-29cd-4a10-bc43-2185e79ae2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:54.299946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86488b-29cd-4a10-bc43-2185e79ae2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:54.300236 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86488b-29cd-4a10-bc43-2185e79ae2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:54.300521 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86488b-29cd-4a10-bc43-2185e79ae2d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:54.301230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86488b-29cd-4a10-bc43-2185e79ae2d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:54.303191 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f86488b-29cd-4a10-bc43-2185e79ae2d9 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:54.324607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1089/2170] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:54 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:13:54.328547 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715043f-fac5-4ad5-bcdd-e43c5c743f2e admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:54.339746 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715043f-fac5-4ad5-bcdd-e43c5c743f2e admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:54.345224 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b88d13-8d2f-4b15-8514-5d330969fc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:54.346323 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b88d13-8d2f-4b15-8514-5d330969fc5f admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:54.346807 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b88d13-8d2f-4b15-8514-5d330969fc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:54.347156 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b88d13-8d2f-4b15-8514-5d330969fc5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:54.347966 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b88d13-8d2f-4b15-8514-5d330969fc5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:54.349541 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b88d13-8d2f-4b15-8514-5d330969fc5f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:54.361288 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1082/2171] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:54.365197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86488b-29cd-4a10-bc43-2185e79ae2d9 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:54.376224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86488b-29cd-4a10-bc43-2185e79ae2d9 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:54.381005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473bf4ac-6139-45f6-b977-9454b8782a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:54.381716 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bf4ac-6139-45f6-b977-9454b8782a4e admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:54.381986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bf4ac-6139-45f6-b977-9454b8782a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:54.382281 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473bf4ac-6139-45f6-b977-9454b8782a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:54.383413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bf4ac-6139-45f6-b977-9454b8782a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:54.385005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473bf4ac-6139-45f6-b977-9454b8782a4e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:54.395050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1090/2172] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:54.400052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b88d13-8d2f-4b15-8514-5d330969fc5f admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:54.410134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b88d13-8d2f-4b15-8514-5d330969fc5f admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:54.415459 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e86924-bc83-4e16-8a8b-5b586c170918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:54.416364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e86924-bc83-4e16-8a8b-5b586c170918 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:54.416653 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e86924-bc83-4e16-8a8b-5b586c170918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:54.416936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e86924-bc83-4e16-8a8b-5b586c170918 admin admin] PATH_INFO: `/v3/projects/ce1b011c885b4d35af18a4c1587c01c6/users/d8544eaefee545baaf05b6af9bbb83f8/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:54.431410 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e86924-bc83-4e16-8a8b-5b586c170918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce1b011c885b4d35af18a4c1587c01c6, user_id=d8544eaefee545baaf05b6af9bbb83f8, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:54.434097 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e86924-bc83-4e16-8a8b-5b586c170918 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:54.446588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1083/2173] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:54 2022] PUT /identity/v3/projects/ce1b011c885b4d35af18a4c1587c01c6/users/d8544eaefee545baaf05b6af9bbb83f8/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:54.457954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c13201-b7c4-4c4b-bd15-ca5d017d1056 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:54.458253 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c13201-b7c4-4c4b-bd15-ca5d017d1056 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:54.458538 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c13201-b7c4-4c4b-bd15-ca5d017d1056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:54.479223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:54.488898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-34c13201-b7c4-4c4b-bd15-ca5d017d1056 None None] MFA Rules not processed for user `d8544eaefee545baaf05b6af9bbb83f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:54.490129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-34c13201-b7c4-4c4b-bd15-ca5d017d1056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:54.555643 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1091/2174] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:54.560138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e86924-bc83-4e16-8a8b-5b586c170918 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:54.622485 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e86924-bc83-4e16-8a8b-5b586c170918 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:54.627415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50246f-052d-483d-bbad-d858a8c0ae43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:54.628007 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50246f-052d-483d-bbad-d858a8c0ae43 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:54.628288 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50246f-052d-483d-bbad-d858a8c0ae43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:54.628563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50246f-052d-483d-bbad-d858a8c0ae43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:54.629241 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50246f-052d-483d-bbad-d858a8c0ae43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:54.631177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50246f-052d-483d-bbad-d858a8c0ae43 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:54.659427 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1084/2175] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:54 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:54.663529 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c13201-b7c4-4c4b-bd15-ca5d017d1056 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:54.674461 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c13201-b7c4-4c4b-bd15-ca5d017d1056 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:54.734665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dad044-da1d-47b1-b9ce-733aa726a5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:54.735368 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dad044-da1d-47b1-b9ce-733aa726a5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:54.735651 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dad044-da1d-47b1-b9ce-733aa726a5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:54.735935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dad044-da1d-47b1-b9ce-733aa726a5c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:54.736646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dad044-da1d-47b1-b9ce-733aa726a5c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:54.738542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dad044-da1d-47b1-b9ce-733aa726a5c0 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:54.759663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1092/2176] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:54 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:54.763594 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50246f-052d-483d-bbad-d858a8c0ae43 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:54.773883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50246f-052d-483d-bbad-d858a8c0ae43 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:54.778511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b6ba7-61c3-405f-a4db-d5a53d044fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:54.779218 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b6ba7-61c3-405f-a4db-d5a53d044fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:54.779505 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b6ba7-61c3-405f-a4db-d5a53d044fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:54.779792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b6ba7-61c3-405f-a4db-d5a53d044fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:54.780552 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b6ba7-61c3-405f-a4db-d5a53d044fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:54.782156 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b6ba7-61c3-405f-a4db-d5a53d044fc9 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:54.794114 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1085/2177] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:54.798252 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dad044-da1d-47b1-b9ce-733aa726a5c0 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:54.809151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dad044-da1d-47b1-b9ce-733aa726a5c0 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:54.813867 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38593a-9fdb-412e-aee1-ff25dcca28cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:54.814770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38593a-9fdb-412e-aee1-ff25dcca28cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:54.815058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38593a-9fdb-412e-aee1-ff25dcca28cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:54.815330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38593a-9fdb-412e-aee1-ff25dcca28cb admin admin] PATH_INFO: `/v3/projects/a488f82ad49f42b8af0d9b664a96140a/users/1bc4afc79c084138a85a26c7114ab7c7/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:54.830126 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38593a-9fdb-412e-aee1-ff25dcca28cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a488f82ad49f42b8af0d9b664a96140a, user_id=1bc4afc79c084138a85a26c7114ab7c7, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:54.832802 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38593a-9fdb-412e-aee1-ff25dcca28cb admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:54.844885 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1093/2178] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:54 2022] PUT /identity/v3/projects/a488f82ad49f42b8af0d9b664a96140a/users/1bc4afc79c084138a85a26c7114ab7c7/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:54.848755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b6ba7-61c3-405f-a4db-d5a53d044fc9 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:54.859778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b6ba7-61c3-405f-a4db-d5a53d044fc9 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:54.897155 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:54.917585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f45a92-0ba0-411d-b77d-151119723990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:54.918257 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f45a92-0ba0-411d-b77d-151119723990 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:54.918647 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f45a92-0ba0-411d-b77d-151119723990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:54.918964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f45a92-0ba0-411d-b77d-151119723990 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:54.919697 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f45a92-0ba0-411d-b77d-151119723990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:54.921312 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f45a92-0ba0-411d-b77d-151119723990 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:54.931228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1086/2179] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:13:54.935458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38593a-9fdb-412e-aee1-ff25dcca28cb admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:54.946149 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38593a-9fdb-412e-aee1-ff25dcca28cb admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:54.951134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ec8f5-5d30-4dfb-b775-135d13a5d1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:54.952014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ec8f5-5d30-4dfb-b775-135d13a5d1eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:54.952325 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ec8f5-5d30-4dfb-b775-135d13a5d1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:54.952601 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ec8f5-5d30-4dfb-b775-135d13a5d1eb admin admin] PATH_INFO: `/v3/projects/a488f82ad49f42b8af0d9b664a96140a/users/1bc4afc79c084138a85a26c7114ab7c7/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:54.954583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ec8f5-5d30-4dfb-b775-135d13a5d1eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a488f82ad49f42b8af0d9b664a96140a, user_id=1bc4afc79c084138a85a26c7114ab7c7, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:54.957373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ec8f5-5d30-4dfb-b775-135d13a5d1eb admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:54.968983 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1094/2180] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:54 2022] PUT /identity/v3/projects/a488f82ad49f42b8af0d9b664a96140a/users/1bc4afc79c084138a85a26c7114ab7c7/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:13:54.980874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40f5aa-86ee-425d-bf30-0443a6761237 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:54.980874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40f5aa-86ee-425d-bf30-0443a6761237 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:54.980874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40f5aa-86ee-425d-bf30-0443a6761237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:55.006442 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-5b40f5aa-86ee-425d-bf30-0443a6761237 None None] MFA Rules not processed for user `1bc4afc79c084138a85a26c7114ab7c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:55.007760 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5b40f5aa-86ee-425d-bf30-0443a6761237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:55.075627 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1087/2181] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:54 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:55.087445 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ec8f5-5d30-4dfb-b775-135d13a5d1eb admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:55.089174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d24ec8f5-5d30-4dfb-b775-135d13a5d1eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:55.152958 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ec8f5-5d30-4dfb-b775-135d13a5d1eb admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:55.156740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850cd75-4309-4d9f-80c4-0d8b0e53de63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:55.157397 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850cd75-4309-4d9f-80c4-0d8b0e53de63 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:55.157687 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850cd75-4309-4d9f-80c4-0d8b0e53de63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:55.158067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850cd75-4309-4d9f-80c4-0d8b0e53de63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:55.163314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:55.230826 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850cd75-4309-4d9f-80c4-0d8b0e53de63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:55.232879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850cd75-4309-4d9f-80c4-0d8b0e53de63 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:55.242860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1095/2182] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:55.472851 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40f5aa-86ee-425d-bf30-0443a6761237 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:55.483140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40f5aa-86ee-425d-bf30-0443a6761237 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:55.487224 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81eafb-3000-4e93-9ccc-e193ec4c5cd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:55.488244 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81eafb-3000-4e93-9ccc-e193ec4c5cd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:55.488689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81eafb-3000-4e93-9ccc-e193ec4c5cd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:55.489022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81eafb-3000-4e93-9ccc-e193ec4c5cd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:55.490948 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-1b81eafb-3000-4e93-9ccc-e193ec4c5cd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:55.580600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81eafb-3000-4e93-9ccc-e193ec4c5cd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:55.582035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81eafb-3000-4e93-9ccc-e193ec4c5cd8 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:55.600452 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1088/2183] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:57.544962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850cd75-4309-4d9f-80c4-0d8b0e53de63 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:57.546508 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c850cd75-4309-4d9f-80c4-0d8b0e53de63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:57.615866 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850cd75-4309-4d9f-80c4-0d8b0e53de63 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:57.620996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297f4f4-b4ac-4556-b0f9-c9d6e776a4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:57.621904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297f4f4-b4ac-4556-b0f9-c9d6e776a4e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:57.622189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297f4f4-b4ac-4556-b0f9-c9d6e776a4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:57.622462 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297f4f4-b4ac-4556-b0f9-c9d6e776a4e9 admin admin] PATH_INFO: `/v3/users/641ade550b8846efb1aff2adb151ac89` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:57.623819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297f4f4-b4ac-4556-b0f9-c9d6e776a4e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=641ade550b8846efb1aff2adb151ac89)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:57.625761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297f4f4-b4ac-4556-b0f9-c9d6e776a4e9 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:57.687096 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2297f4f4-b4ac-4556-b0f9-c9d6e776a4e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '641ade550b8846efb1aff2adb151ac89'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:57.700158 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2297f4f4-b4ac-4556-b0f9-c9d6e776a4e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '641ade550b8846efb1aff2adb151ac89'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:57.707305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2297f4f4-b4ac-4556-b0f9-c9d6e776a4e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '641ade550b8846efb1aff2adb151ac89'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:57.708074 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2297f4f4-b4ac-4556-b0f9-c9d6e776a4e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '641ade550b8846efb1aff2adb151ac89'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:57.715993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2297f4f4-b4ac-4556-b0f9-c9d6e776a4e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '641ade550b8846efb1aff2adb151ac89'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:57.723707 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1096/2184] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:57 2022] DELETE /identity/v3/users/641ade550b8846efb1aff2adb151ac89 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:57.729327 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bf3f0-c51e-40b8-bb53-b5ff87373398 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:57.729792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bf3f0-c51e-40b8-bb53-b5ff87373398 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:57.729880 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bf3f0-c51e-40b8-bb53-b5ff87373398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:57.749196 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-b12bf3f0-c51e-40b8-bb53-b5ff87373398 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:57.750500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b12bf3f0-c51e-40b8-bb53-b5ff87373398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:57.808108 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:57.822401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1089/2185] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:57.832543 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297f4f4-b4ac-4556-b0f9-c9d6e776a4e9 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:57.834595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2297f4f4-b4ac-4556-b0f9-c9d6e776a4e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:57.861920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bf3f0-c51e-40b8-bb53-b5ff87373398 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:57.863596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b12bf3f0-c51e-40b8-bb53-b5ff87373398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:57.881555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:57.922933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297f4f4-b4ac-4556-b0f9-c9d6e776a4e9 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:57.927679 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38892303-7eb1-42b0-90ca-559054c4206c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:57.928397 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38892303-7eb1-42b0-90ca-559054c4206c service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:57.928689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38892303-7eb1-42b0-90ca-559054c4206c service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:57.929902 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38892303-7eb1-42b0-90ca-559054c4206c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:57.942592 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38892303-7eb1-42b0-90ca-559054c4206c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:57.944065 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38892303-7eb1-42b0-90ca-559054c4206c service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:57.946943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bf3f0-c51e-40b8-bb53-b5ff87373398 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:57.963261 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f4d3e-c191-47ff-a582-b0f6558593b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:57.963967 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f4d3e-c191-47ff-a582-b0f6558593b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:57.964256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f4d3e-c191-47ff-a582-b0f6558593b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:57.964588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f4d3e-c191-47ff-a582-b0f6558593b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:57.965662 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1097/2186] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:57.967924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3a4f4d3e-c191-47ff-a582-b0f6558593b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:58.045018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f4d3e-c191-47ff-a582-b0f6558593b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:58.046652 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f4d3e-c191-47ff-a582-b0f6558593b6 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:58.058512 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1090/2187] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:58.513040 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38892303-7eb1-42b0-90ca-559054c4206c service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:58.514725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-38892303-7eb1-42b0-90ca-559054c4206c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:58.534474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38892303-7eb1-42b0-90ca-559054c4206c service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:58.543724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe055e7-b2d0-4ed3-999c-fd0074e45312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:58.544786 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe055e7-b2d0-4ed3-999c-fd0074e45312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:58.545146 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe055e7-b2d0-4ed3-999c-fd0074e45312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:58.545856 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe055e7-b2d0-4ed3-999c-fd0074e45312 admin admin] PATH_INFO: `/v3/projects/5289f735719544f2a758895a6d4bf3c0` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:58.549675 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe055e7-b2d0-4ed3-999c-fd0074e45312 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5289f735719544f2a758895a6d4bf3c0)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:58.551820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe055e7-b2d0-4ed3-999c-fd0074e45312 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:58.619912 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6fe055e7-b2d0-4ed3-999c-fd0074e45312 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5289f735719544f2a758895a6d4bf3c0', 'request_id': 'req-6fe055e7-b2d0-4ed3-999c-fd0074e45312'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:58.626342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6fe055e7-b2d0-4ed3-999c-fd0074e45312 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5289f735719544f2a758895a6d4bf3c0', 'request_id': 'req-6fe055e7-b2d0-4ed3-999c-fd0074e45312'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:58.633490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6fe055e7-b2d0-4ed3-999c-fd0074e45312 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5289f735719544f2a758895a6d4bf3c0', 'request_id': 'req-6fe055e7-b2d0-4ed3-999c-fd0074e45312'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:58.653105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6fe055e7-b2d0-4ed3-999c-fd0074e45312 admin admin] The token cache is being invalidate because project 5289f735719544f2a758895a6d4bf3c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:13:58.653398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6fe055e7-b2d0-4ed3-999c-fd0074e45312 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:58.654119 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6fe055e7-b2d0-4ed3-999c-fd0074e45312 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:58.656200 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1098/2188] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:13:58 2022] DELETE /identity/v3/projects/5289f735719544f2a758895a6d4bf3c0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:58.661259 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726f572-3d3e-47d9-a880-07285817e0e8 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:58.661748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726f572-3d3e-47d9-a880-07285817e0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:58.662042 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726f572-3d3e-47d9-a880-07285817e0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:58.697910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-d726f572-3d3e-47d9-a880-07285817e0e8 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:58.701685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d726f572-3d3e-47d9-a880-07285817e0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:58.820641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1091/2189] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:58.829465 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c1eae-543e-4a79-b5f8-107fe64793e7 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:58.829731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c1eae-543e-4a79-b5f8-107fe64793e7 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:58.830137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c1eae-543e-4a79-b5f8-107fe64793e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:58.849900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-975c1eae-543e-4a79-b5f8-107fe64793e7 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:58.850947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-975c1eae-543e-4a79-b5f8-107fe64793e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:58.859497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1099/2190] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:13:58.863753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726f572-3d3e-47d9-a880-07285817e0e8 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:58.874080 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726f572-3d3e-47d9-a880-07285817e0e8 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:58.878938 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d16907-e3de-4796-8c5b-143a23d332a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:58.879665 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d16907-e3de-4796-8c5b-143a23d332a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:58.880018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d16907-e3de-4796-8c5b-143a23d332a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:58.880361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d16907-e3de-4796-8c5b-143a23d332a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:58.881161 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d16907-e3de-4796-8c5b-143a23d332a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:13:58.881535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d16907-e3de-4796-8c5b-143a23d332a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:58.883702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d16907-e3de-4796-8c5b-143a23d332a4 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:58.903443 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1092/2191] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:13:58.903595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c1eae-543e-4a79-b5f8-107fe64793e7 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:58.903595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-975c1eae-543e-4a79-b5f8-107fe64793e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:58.904458 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d16907-e3de-4796-8c5b-143a23d332a4 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:58.914705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d16907-e3de-4796-8c5b-143a23d332a4 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:58.919299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85104ec7-a31a-4b63-bcf9-98f884caf318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:58.919943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85104ec7-a31a-4b63-bcf9-98f884caf318 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:58.920227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85104ec7-a31a-4b63-bcf9-98f884caf318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:58.920506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85104ec7-a31a-4b63-bcf9-98f884caf318 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:58.921189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85104ec7-a31a-4b63-bcf9-98f884caf318 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:58.923124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85104ec7-a31a-4b63-bcf9-98f884caf318 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:58.951478 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1093/2192] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:13:58 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:13:58.955612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85104ec7-a31a-4b63-bcf9-98f884caf318 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:58.961013 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c1eae-543e-4a79-b5f8-107fe64793e7 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:58.974200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:59.013018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3fbed-47c8-4b94-beac-14db9b75fa1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:59.013674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3fbed-47c8-4b94-beac-14db9b75fa1b service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:59.013958 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3fbed-47c8-4b94-beac-14db9b75fa1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:59.014238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3fbed-47c8-4b94-beac-14db9b75fa1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:59.016102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c2d3fbed-47c8-4b94-beac-14db9b75fa1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:59.021139 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85104ec7-a31a-4b63-bcf9-98f884caf318 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:59.026662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1bd51-5485-44c8-b90b-d3fbea074230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:59.027369 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1bd51-5485-44c8-b90b-d3fbea074230 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:59.027654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1bd51-5485-44c8-b90b-d3fbea074230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:59.027938 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1bd51-5485-44c8-b90b-d3fbea074230 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:59.028631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1bd51-5485-44c8-b90b-d3fbea074230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:59.030106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:59.030569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1bd51-5485-44c8-b90b-d3fbea074230 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:59.051104 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1094/2193] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:13:58 2022] POST /identity/v3/users => generated 281 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:13:59.055358 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1bd51-5485-44c8-b90b-d3fbea074230 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:59.065433 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1bd51-5485-44c8-b90b-d3fbea074230 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:59.071209 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6cf7f-0411-4c70-891b-6e5929f281a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:59.071979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6cf7f-0411-4c70-891b-6e5929f281a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:59.072362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6cf7f-0411-4c70-891b-6e5929f281a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:59.072723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6cf7f-0411-4c70-891b-6e5929f281a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:59.073830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6cf7f-0411-4c70-891b-6e5929f281a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:59.075597 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6cf7f-0411-4c70-891b-6e5929f281a5 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:59.084578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3fbed-47c8-4b94-beac-14db9b75fa1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:59.086950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1095/2194] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:59.087091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3fbed-47c8-4b94-beac-14db9b75fa1b service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:59.090844 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6cf7f-0411-4c70-891b-6e5929f281a5 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:59.096684 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1100/2195] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:59.101147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6cf7f-0411-4c70-891b-6e5929f281a5 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:59.106188 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0c81a-7bdb-42ba-ac54-6025892ca887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:59.106921 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0c81a-7bdb-42ba-ac54-6025892ca887 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:59.107210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0c81a-7bdb-42ba-ac54-6025892ca887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:59.107496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0c81a-7bdb-42ba-ac54-6025892ca887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:59.108234 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0c81a-7bdb-42ba-ac54-6025892ca887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:59.109849 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0c81a-7bdb-42ba-ac54-6025892ca887 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:59.120805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1096/2196] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:13:59.125069 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3fbed-47c8-4b94-beac-14db9b75fa1b service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:59.136163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3fbed-47c8-4b94-beac-14db9b75fa1b service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:59.140746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abe770-7802-49f5-a08e-77733e07e381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:59.141627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abe770-7802-49f5-a08e-77733e07e381 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:59.142032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abe770-7802-49f5-a08e-77733e07e381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:59.142316 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abe770-7802-49f5-a08e-77733e07e381 admin admin] PATH_INFO: `/v3/projects/0577bb7d73b245248897db8026d7c658/users/58f8f7d465bb465bb7f024a7be8ccf39/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:59.163175 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abe770-7802-49f5-a08e-77733e07e381 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0577bb7d73b245248897db8026d7c658, user_id=58f8f7d465bb465bb7f024a7be8ccf39, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:59.165935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abe770-7802-49f5-a08e-77733e07e381 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:59.180291 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1101/2197] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:13:59 2022] PUT /identity/v3/projects/0577bb7d73b245248897db8026d7c658/users/58f8f7d465bb465bb7f024a7be8ccf39/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:59.191760 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb0ce1-9e58-439d-b90e-889cd9d77d58 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:59.192949 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb0ce1-9e58-439d-b90e-889cd9d77d58 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:59.193264 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb0ce1-9e58-439d-b90e-889cd9d77d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:59.226726 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-eaeb0ce1-9e58-439d-b90e-889cd9d77d58 None None] MFA Rules not processed for user `58f8f7d465bb465bb7f024a7be8ccf39`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:59.228020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-eaeb0ce1-9e58-439d-b90e-889cd9d77d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:59.300297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1097/2198] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:59.334722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abe770-7802-49f5-a08e-77733e07e381 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:59.336464 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e2abe770-7802-49f5-a08e-77733e07e381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:59.396274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abe770-7802-49f5-a08e-77733e07e381 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:59.400574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d5a3d-e537-4b85-b1b9-8080c363247b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:59.401223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d5a3d-e537-4b85-b1b9-8080c363247b service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:59.401506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d5a3d-e537-4b85-b1b9-8080c363247b service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:59.401785 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d5a3d-e537-4b85-b1b9-8080c363247b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:59.411137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d5a3d-e537-4b85-b1b9-8080c363247b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:59.412548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d5a3d-e537-4b85-b1b9-8080c363247b service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:59.422186 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1102/2199] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:13:59.435950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb0ce1-9e58-439d-b90e-889cd9d77d58 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:59.513538 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb0ce1-9e58-439d-b90e-889cd9d77d58 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:59.518860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396e43d-f761-4063-8c94-3d51a0bfd1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:59.519984 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396e43d-f761-4063-8c94-3d51a0bfd1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:59.520377 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396e43d-f761-4063-8c94-3d51a0bfd1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:59.520885 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396e43d-f761-4063-8c94-3d51a0bfd1bf admin admin] PATH_INFO: `/v3/users/58f8f7d465bb465bb7f024a7be8ccf39` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:59.522202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396e43d-f761-4063-8c94-3d51a0bfd1bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=58f8f7d465bb465bb7f024a7be8ccf39)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:59.525610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396e43d-f761-4063-8c94-3d51a0bfd1bf admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:59.554318 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:13:59.595674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-3396e43d-f761-4063-8c94-3d51a0bfd1bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58f8f7d465bb465bb7f024a7be8ccf39'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:59.608505 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-3396e43d-f761-4063-8c94-3d51a0bfd1bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58f8f7d465bb465bb7f024a7be8ccf39'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:59.615575 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-3396e43d-f761-4063-8c94-3d51a0bfd1bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58f8f7d465bb465bb7f024a7be8ccf39'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:59.616295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-3396e43d-f761-4063-8c94-3d51a0bfd1bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58f8f7d465bb465bb7f024a7be8ccf39'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:59.624751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-3396e43d-f761-4063-8c94-3d51a0bfd1bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58f8f7d465bb465bb7f024a7be8ccf39'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:13:59.631156 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1098/2200] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:13:59 2022] DELETE /identity/v3/users/58f8f7d465bb465bb7f024a7be8ccf39 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:13:59.635915 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9833927-4cdf-4fb6-b607-235add1548ea None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:59.636489 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9833927-4cdf-4fb6-b607-235add1548ea None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:59.636489 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9833927-4cdf-4fb6-b607-235add1548ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:59.653992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-b9833927-4cdf-4fb6-b607-235add1548ea None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:13:59.655232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b9833927-4cdf-4fb6-b607-235add1548ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:59.715038 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1103/2201] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:13:59.724680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396e43d-f761-4063-8c94-3d51a0bfd1bf admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:13:59.726152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3396e43d-f761-4063-8c94-3d51a0bfd1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:13:59.783880 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396e43d-f761-4063-8c94-3d51a0bfd1bf admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:13:59.787473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cbc2fe-d350-46d4-b3e1-411e093e5f07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:13:59.788117 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbc2fe-d350-46d4-b3e1-411e093e5f07 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:13:59.788404 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbc2fe-d350-46d4-b3e1-411e093e5f07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:13:59.788721 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbc2fe-d350-46d4-b3e1-411e093e5f07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:13:59.797785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cbc2fe-d350-46d4-b3e1-411e093e5f07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:13:59.799271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cbc2fe-d350-46d4-b3e1-411e093e5f07 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:13:59.808892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1099/2202] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:13:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:14:00.090186 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9833927-4cdf-4fb6-b607-235add1548ea None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:00.091739 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b9833927-4cdf-4fb6-b607-235add1548ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:00.104567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9833927-4cdf-4fb6-b607-235add1548ea None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:00.109136 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b6d48-d827-4445-9e59-d20a12ce92c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:00.109997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b6d48-d827-4445-9e59-d20a12ce92c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:00.110273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b6d48-d827-4445-9e59-d20a12ce92c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:00.110547 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b6d48-d827-4445-9e59-d20a12ce92c2 admin admin] PATH_INFO: `/v3/projects/0577bb7d73b245248897db8026d7c658` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:00.111751 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b6d48-d827-4445-9e59-d20a12ce92c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0577bb7d73b245248897db8026d7c658)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:00.113756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b6d48-d827-4445-9e59-d20a12ce92c2 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:00.139272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:00.161869 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-718b6d48-d827-4445-9e59-d20a12ce92c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0577bb7d73b245248897db8026d7c658', 'request_id': 'req-718b6d48-d827-4445-9e59-d20a12ce92c2'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:00.167607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-718b6d48-d827-4445-9e59-d20a12ce92c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0577bb7d73b245248897db8026d7c658', 'request_id': 'req-718b6d48-d827-4445-9e59-d20a12ce92c2'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:00.173558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-718b6d48-d827-4445-9e59-d20a12ce92c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0577bb7d73b245248897db8026d7c658', 'request_id': 'req-718b6d48-d827-4445-9e59-d20a12ce92c2'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:00.186698 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-718b6d48-d827-4445-9e59-d20a12ce92c2 admin admin] The token cache is being invalidate because project 0577bb7d73b245248897db8026d7c658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:14:00.187023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-718b6d48-d827-4445-9e59-d20a12ce92c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:00.187652 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-718b6d48-d827-4445-9e59-d20a12ce92c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:00.189198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1104/2203] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:14:00 2022] DELETE /identity/v3/projects/0577bb7d73b245248897db8026d7c658 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:00.194061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231206eb-bb5d-4675-843e-28fe3467a642 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:00.194511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231206eb-bb5d-4675-843e-28fe3467a642 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:00.194642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231206eb-bb5d-4675-843e-28fe3467a642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:00.213070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-231206eb-bb5d-4675-843e-28fe3467a642 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:00.214269 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-231206eb-bb5d-4675-843e-28fe3467a642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:00.276240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1100/2204] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:00.284835 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10801917-4b43-4306-a34c-8e78ee3c50ab None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:00.285167 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10801917-4b43-4306-a34c-8e78ee3c50ab None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:00.285454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10801917-4b43-4306-a34c-8e78ee3c50ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:00.303722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-10801917-4b43-4306-a34c-8e78ee3c50ab None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:00.306314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-10801917-4b43-4306-a34c-8e78ee3c50ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:00.314543 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1105/2205] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:00.318651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231206eb-bb5d-4675-843e-28fe3467a642 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:00.329561 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231206eb-bb5d-4675-843e-28fe3467a642 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:00.334184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5984b57-0bd6-4109-8eb2-a7894be4bbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:00.334809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5984b57-0bd6-4109-8eb2-a7894be4bbcb admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:00.335094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5984b57-0bd6-4109-8eb2-a7894be4bbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:00.335405 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5984b57-0bd6-4109-8eb2-a7894be4bbcb admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:00.336383 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5984b57-0bd6-4109-8eb2-a7894be4bbcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:14:00.336695 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5984b57-0bd6-4109-8eb2-a7894be4bbcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:00.338307 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5984b57-0bd6-4109-8eb2-a7894be4bbcb admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:00.349645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1101/2206] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:14:00.354315 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10801917-4b43-4306-a34c-8e78ee3c50ab None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:00.365400 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10801917-4b43-4306-a34c-8e78ee3c50ab None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:00.371302 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfde54-0177-47b8-b78f-fefd85193df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:00.371961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfde54-0177-47b8-b78f-fefd85193df9 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:00.372246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfde54-0177-47b8-b78f-fefd85193df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:00.372524 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfde54-0177-47b8-b78f-fefd85193df9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:00.373216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfde54-0177-47b8-b78f-fefd85193df9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:00.375160 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfde54-0177-47b8-b78f-fefd85193df9 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:00.402923 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1106/2207] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:14:00 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:14:00.407029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5984b57-0bd6-4109-8eb2-a7894be4bbcb admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:00.416846 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5984b57-0bd6-4109-8eb2-a7894be4bbcb admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:00.435387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:00.477076 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248d3c7-4418-4394-84cf-38babd1ac876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:00.477745 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248d3c7-4418-4394-84cf-38babd1ac876 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:00.478031 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248d3c7-4418-4394-84cf-38babd1ac876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:00.478308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248d3c7-4418-4394-84cf-38babd1ac876 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:00.479039 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248d3c7-4418-4394-84cf-38babd1ac876 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:00.480962 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248d3c7-4418-4394-84cf-38babd1ac876 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:00.500933 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1102/2208] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:14:00 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:14:00.505006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfde54-0177-47b8-b78f-fefd85193df9 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:00.516394 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfde54-0177-47b8-b78f-fefd85193df9 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:00.520971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013a4d2-56e3-4079-ad64-b35acf30a2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:00.521626 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013a4d2-56e3-4079-ad64-b35acf30a2ed admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:00.521911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013a4d2-56e3-4079-ad64-b35acf30a2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:00.522192 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013a4d2-56e3-4079-ad64-b35acf30a2ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:00.523984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d013a4d2-56e3-4079-ad64-b35acf30a2ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:00.525605 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d013a4d2-56e3-4079-ad64-b35acf30a2ed admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:00.536835 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1107/2209] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:14:00.540957 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248d3c7-4418-4394-84cf-38babd1ac876 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:00.552090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248d3c7-4418-4394-84cf-38babd1ac876 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:00.557459 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb982da-474a-44ba-9615-f2fc538a7b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:00.558155 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb982da-474a-44ba-9615-f2fc538a7b6e admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:00.558442 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb982da-474a-44ba-9615-f2fc538a7b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:00.558769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb982da-474a-44ba-9615-f2fc538a7b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:00.559507 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb982da-474a-44ba-9615-f2fc538a7b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:00.561135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb982da-474a-44ba-9615-f2fc538a7b6e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:00.575704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1103/2210] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:14:00.577276 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013a4d2-56e3-4079-ad64-b35acf30a2ed admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:00.588278 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013a4d2-56e3-4079-ad64-b35acf30a2ed admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:00.592938 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4371cd-5069-43a1-8be5-d00349da66a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:00.593810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4371cd-5069-43a1-8be5-d00349da66a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:00.594105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4371cd-5069-43a1-8be5-d00349da66a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:00.594383 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4371cd-5069-43a1-8be5-d00349da66a8 admin admin] PATH_INFO: `/v3/projects/05578c66d5b14af29cc0d860ed978fbf/users/7fed4d6dad5b4a06ae84a26b2aab9850/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:00.609592 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4371cd-5069-43a1-8be5-d00349da66a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05578c66d5b14af29cc0d860ed978fbf, user_id=7fed4d6dad5b4a06ae84a26b2aab9850, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:00.612245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4371cd-5069-43a1-8be5-d00349da66a8 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:00.625452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1108/2211] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:14:00 2022] PUT /identity/v3/projects/05578c66d5b14af29cc0d860ed978fbf/users/7fed4d6dad5b4a06ae84a26b2aab9850/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:00.630464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa8eec-c1e0-4d3b-ade5-71c5a058bbb0 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:00.630794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa8eec-c1e0-4d3b-ade5-71c5a058bbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:00.631084 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa8eec-c1e0-4d3b-ade5-71c5a058bbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:00.650370 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-f4fa8eec-c1e0-4d3b-ade5-71c5a058bbb0 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:00.651648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f4fa8eec-c1e0-4d3b-ade5-71c5a058bbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:00.712132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1104/2212] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:00.721770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4371cd-5069-43a1-8be5-d00349da66a8 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:00.723536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-db4371cd-5069-43a1-8be5-d00349da66a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:00.782543 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4371cd-5069-43a1-8be5-d00349da66a8 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:00.786306 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802ca4a-80b3-4185-8626-8393fdce530a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:00.786985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802ca4a-80b3-4185-8626-8393fdce530a service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:00.787273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802ca4a-80b3-4185-8626-8393fdce530a service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:00.787554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802ca4a-80b3-4185-8626-8393fdce530a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:00.797017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802ca4a-80b3-4185-8626-8393fdce530a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:00.798448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802ca4a-80b3-4185-8626-8393fdce530a service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:00.808596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1109/2213] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:14:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:14:01.423245 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca19a9e-ce2e-4738-9000-3ffcaf7bc8c2 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:01.423774 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca19a9e-ce2e-4738-9000-3ffcaf7bc8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:01.424073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca19a9e-ce2e-4738-9000-3ffcaf7bc8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:01.451001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-6ca19a9e-ce2e-4738-9000-3ffcaf7bc8c2 None None] MFA Rules not processed for user `7fed4d6dad5b4a06ae84a26b2aab9850`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:01.452212 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6ca19a9e-ce2e-4738-9000-3ffcaf7bc8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:01.518252 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1105/2214] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:01.529295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802ca4a-80b3-4185-8626-8393fdce530a service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:01.530725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e802ca4a-80b3-4185-8626-8393fdce530a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:01.593455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802ca4a-80b3-4185-8626-8393fdce530a service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:01.597712 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacb7fdd-9eca-4649-871c-36aef2531b84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:01.598354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb7fdd-9eca-4649-871c-36aef2531b84 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:01.598637 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb7fdd-9eca-4649-871c-36aef2531b84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:01.598947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacb7fdd-9eca-4649-871c-36aef2531b84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:01.605097 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:01.609391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb7fdd-9eca-4649-871c-36aef2531b84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:01.611079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacb7fdd-9eca-4649-871c-36aef2531b84 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:01.623162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1110/2215] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:14:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:14:02.909722 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca19a9e-ce2e-4738-9000-3ffcaf7bc8c2 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:02.911197 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6ca19a9e-ce2e-4738-9000-3ffcaf7bc8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:02.924978 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca19a9e-ce2e-4738-9000-3ffcaf7bc8c2 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:02.929726 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005af63-f434-4963-aa46-3fd54a08afe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:02.930592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005af63-f434-4963-aa46-3fd54a08afe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:02.930912 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005af63-f434-4963-aa46-3fd54a08afe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:02.931202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005af63-f434-4963-aa46-3fd54a08afe8 admin admin] PATH_INFO: `/v3/users/3e6e86c1ff9d430bac353aa8bd4f3418` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:02.932447 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005af63-f434-4963-aa46-3fd54a08afe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e6e86c1ff9d430bac353aa8bd4f3418)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:02.934407 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6005af63-f434-4963-aa46-3fd54a08afe8 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:02.964120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:02.984871 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6005af63-f434-4963-aa46-3fd54a08afe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e6e86c1ff9d430bac353aa8bd4f3418'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:02.997326 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6005af63-f434-4963-aa46-3fd54a08afe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e6e86c1ff9d430bac353aa8bd4f3418'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:03.006646 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6005af63-f434-4963-aa46-3fd54a08afe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e6e86c1ff9d430bac353aa8bd4f3418'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:03.008028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6005af63-f434-4963-aa46-3fd54a08afe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e6e86c1ff9d430bac353aa8bd4f3418'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:03.019461 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6005af63-f434-4963-aa46-3fd54a08afe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e6e86c1ff9d430bac353aa8bd4f3418'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:03.037818 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b23e85-1dd2-46b5-a549-4c7d49fafb60 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:03.039311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b23e85-1dd2-46b5-a549-4c7d49fafb60 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:03.040148 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b23e85-1dd2-46b5-a549-4c7d49fafb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:03.042569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1106/2216] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:14:02 2022] DELETE /identity/v3/users/3e6e86c1ff9d430bac353aa8bd4f3418 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:03.047256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d0796-8649-4126-802e-f1bb793c0954 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:03.047717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d0796-8649-4126-802e-f1bb793c0954 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:03.048183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d0796-8649-4126-802e-f1bb793c0954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:03.059771 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-e4b23e85-1dd2-46b5-a549-4c7d49fafb60 None None] MFA Rules not processed for user `7fed4d6dad5b4a06ae84a26b2aab9850`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:03.061434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e4b23e85-1dd2-46b5-a549-4c7d49fafb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:03.066712 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-f85d0796-8649-4126-802e-f1bb793c0954 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:03.067645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f85d0796-8649-4126-802e-f1bb793c0954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:03.130244 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1111/2217] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:14:03.150765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1107/2218] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:03.160748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b23e85-1dd2-46b5-a549-4c7d49fafb60 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:03.162134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e4b23e85-1dd2-46b5-a549-4c7d49fafb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:03.236384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b23e85-1dd2-46b5-a549-4c7d49fafb60 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:03.240305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf5bf7-4f69-4aef-a819-84891d5d4617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:03.240958 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf5bf7-4f69-4aef-a819-84891d5d4617 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:03.241394 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf5bf7-4f69-4aef-a819-84891d5d4617 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:03.242422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf5bf7-4f69-4aef-a819-84891d5d4617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:03.252215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf5bf7-4f69-4aef-a819-84891d5d4617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:03.259696 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf5bf7-4f69-4aef-a819-84891d5d4617 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:03.273488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1112/2219] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:14:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:14:03.860244 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d0796-8649-4126-802e-f1bb793c0954 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:03.861883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f85d0796-8649-4126-802e-f1bb793c0954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:03.887408 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d0796-8649-4126-802e-f1bb793c0954 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:03.893060 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62202d2f-f075-423f-8834-18958f73510e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:03.893947 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62202d2f-f075-423f-8834-18958f73510e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:03.894231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62202d2f-f075-423f-8834-18958f73510e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:03.894510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62202d2f-f075-423f-8834-18958f73510e admin admin] PATH_INFO: `/v3/projects/a1ecb2ef4bdd405b942ca70ffdaba8ea` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:03.895919 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62202d2f-f075-423f-8834-18958f73510e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1ecb2ef4bdd405b942ca70ffdaba8ea)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:03.899024 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62202d2f-f075-423f-8834-18958f73510e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:03.965054 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-62202d2f-f075-423f-8834-18958f73510e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1ecb2ef4bdd405b942ca70ffdaba8ea', 'request_id': 'req-62202d2f-f075-423f-8834-18958f73510e'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:03.971500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-62202d2f-f075-423f-8834-18958f73510e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1ecb2ef4bdd405b942ca70ffdaba8ea', 'request_id': 'req-62202d2f-f075-423f-8834-18958f73510e'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:03.980514 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-62202d2f-f075-423f-8834-18958f73510e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1ecb2ef4bdd405b942ca70ffdaba8ea', 'request_id': 'req-62202d2f-f075-423f-8834-18958f73510e'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:03.998403 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-62202d2f-f075-423f-8834-18958f73510e admin admin] The token cache is being invalidate because project a1ecb2ef4bdd405b942ca70ffdaba8ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:14:03.998974 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-62202d2f-f075-423f-8834-18958f73510e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:03.999848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-62202d2f-f075-423f-8834-18958f73510e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:04.003004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1108/2220] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:14:03 2022] DELETE /identity/v3/projects/a1ecb2ef4bdd405b942ca70ffdaba8ea => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:04.007014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf5bf7-4f69-4aef-a819-84891d5d4617 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:04.025303 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf5bf7-4f69-4aef-a819-84891d5d4617 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:04.110314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c92e88-c681-4cb6-9385-f37c1c259928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:04.112148 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c92e88-c681-4cb6-9385-f37c1c259928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:04.112319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c92e88-c681-4cb6-9385-f37c1c259928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:04.113001 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c92e88-c681-4cb6-9385-f37c1c259928 admin admin] PATH_INFO: `/v3/users/a5a909f4c7614b4dbdc567208d0c3536` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:04.114398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c92e88-c681-4cb6-9385-f37c1c259928 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5a909f4c7614b4dbdc567208d0c3536)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:04.116374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c92e88-c681-4cb6-9385-f37c1c259928 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:04.168330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:04.172654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-75c92e88-c681-4cb6-9385-f37c1c259928 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5a909f4c7614b4dbdc567208d0c3536'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:04.189137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-75c92e88-c681-4cb6-9385-f37c1c259928 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5a909f4c7614b4dbdc567208d0c3536'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:04.197068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-75c92e88-c681-4cb6-9385-f37c1c259928 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5a909f4c7614b4dbdc567208d0c3536'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:04.197813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-75c92e88-c681-4cb6-9385-f37c1c259928 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5a909f4c7614b4dbdc567208d0c3536'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:04.205523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-75c92e88-c681-4cb6-9385-f37c1c259928 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5a909f4c7614b4dbdc567208d0c3536'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:04.212404 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1113/2221] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:14:04 2022] DELETE /identity/v3/users/a5a909f4c7614b4dbdc567208d0c3536 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:04.523405 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62202d2f-f075-423f-8834-18958f73510e admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:04.525468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-62202d2f-f075-423f-8834-18958f73510e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:04.556950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62202d2f-f075-423f-8834-18958f73510e admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:04.604520 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:04.656855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48587f-d918-4445-b41a-a2040f55acf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:04.657849 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48587f-d918-4445-b41a-a2040f55acf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:04.658249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48587f-d918-4445-b41a-a2040f55acf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:04.658539 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d48587f-d918-4445-b41a-a2040f55acf6 admin admin] PATH_INFO: `/v3/projects/37948a13953a42a9bc0c0c6eb3bc420a` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:04.660063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48587f-d918-4445-b41a-a2040f55acf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37948a13953a42a9bc0c0c6eb3bc420a)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:04.662240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d48587f-d918-4445-b41a-a2040f55acf6 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:04.726817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2d48587f-d918-4445-b41a-a2040f55acf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37948a13953a42a9bc0c0c6eb3bc420a', 'request_id': 'req-2d48587f-d918-4445-b41a-a2040f55acf6'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:04.735287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2d48587f-d918-4445-b41a-a2040f55acf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37948a13953a42a9bc0c0c6eb3bc420a', 'request_id': 'req-2d48587f-d918-4445-b41a-a2040f55acf6'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:04.743729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2d48587f-d918-4445-b41a-a2040f55acf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37948a13953a42a9bc0c0c6eb3bc420a', 'request_id': 'req-2d48587f-d918-4445-b41a-a2040f55acf6'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:04.755911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2d48587f-d918-4445-b41a-a2040f55acf6 admin admin] The token cache is being invalidate because project 37948a13953a42a9bc0c0c6eb3bc420a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:14:04.756203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2d48587f-d918-4445-b41a-a2040f55acf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:04.756856 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2d48587f-d918-4445-b41a-a2040f55acf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:04.759187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1109/2222] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:14:04 2022] DELETE /identity/v3/projects/37948a13953a42a9bc0c0c6eb3bc420a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:14:04.763404 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c92e88-c681-4cb6-9385-f37c1c259928 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:04.764875 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-75c92e88-c681-4cb6-9385-f37c1c259928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:04.777582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c92e88-c681-4cb6-9385-f37c1c259928 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:04.834761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81effc-806c-420d-bb3a-36b45741bbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:04.835835 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81effc-806c-420d-bb3a-36b45741bbae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:04.836103 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81effc-806c-420d-bb3a-36b45741bbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:04.836376 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81effc-806c-420d-bb3a-36b45741bbae admin admin] PATH_INFO: `/v3/users/9e0f54e2acbb4906b90357e01aae9a5c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:04.837576 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81effc-806c-420d-bb3a-36b45741bbae admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e0f54e2acbb4906b90357e01aae9a5c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:04.839542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81effc-806c-420d-bb3a-36b45741bbae admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:04.888193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0c81effc-806c-420d-bb3a-36b45741bbae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e0f54e2acbb4906b90357e01aae9a5c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:04.900670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0c81effc-806c-420d-bb3a-36b45741bbae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e0f54e2acbb4906b90357e01aae9a5c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:04.907936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0c81effc-806c-420d-bb3a-36b45741bbae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e0f54e2acbb4906b90357e01aae9a5c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:04.908718 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0c81effc-806c-420d-bb3a-36b45741bbae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e0f54e2acbb4906b90357e01aae9a5c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:04.916467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0c81effc-806c-420d-bb3a-36b45741bbae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e0f54e2acbb4906b90357e01aae9a5c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:04.926337 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1114/2223] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:14:04 2022] DELETE /identity/v3/users/9e0f54e2acbb4906b90357e01aae9a5c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:05.197509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48587f-d918-4445-b41a-a2040f55acf6 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:05.199240 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-2d48587f-d918-4445-b41a-a2040f55acf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:05.213679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d48587f-d918-4445-b41a-a2040f55acf6 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:05.218418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81effc-806c-420d-bb3a-36b45741bbae admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:05.220617 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-0c81effc-806c-420d-bb3a-36b45741bbae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:05.285478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:05.308880 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d217e15-88bb-4c56-a43d-200f95fcac2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:05.309486 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d217e15-88bb-4c56-a43d-200f95fcac2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:05.309846 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d217e15-88bb-4c56-a43d-200f95fcac2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:05.310131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d217e15-88bb-4c56-a43d-200f95fcac2b admin admin] PATH_INFO: `/v3/users/0ca2e554196e46c4ad9100622fed470c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:05.311886 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d217e15-88bb-4c56-a43d-200f95fcac2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ca2e554196e46c4ad9100622fed470c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:05.312548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81effc-806c-420d-bb3a-36b45741bbae admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:05.314181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d217e15-88bb-4c56-a43d-200f95fcac2b admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:05.317336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad09aa-644b-46d6-8b86-a687702b819f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:05.318200 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad09aa-644b-46d6-8b86-a687702b819f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:05.318482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad09aa-644b-46d6-8b86-a687702b819f admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:05.318788 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad09aa-644b-46d6-8b86-a687702b819f admin admin] PATH_INFO: `/v3/projects/be918a9e63cb43928a6ee4206890f323` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:05.320543 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad09aa-644b-46d6-8b86-a687702b819f admin admin] RBAC: Authorizing `identity:delete_project(project_id=be918a9e63cb43928a6ee4206890f323)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:05.322439 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad09aa-644b-46d6-8b86-a687702b819f admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:05.375763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6d217e15-88bb-4c56-a43d-200f95fcac2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ca2e554196e46c4ad9100622fed470c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:05.384855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:05.385735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-15ad09aa-644b-46d6-8b86-a687702b819f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be918a9e63cb43928a6ee4206890f323', 'request_id': 'req-15ad09aa-644b-46d6-8b86-a687702b819f'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:05.391990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-15ad09aa-644b-46d6-8b86-a687702b819f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be918a9e63cb43928a6ee4206890f323', 'request_id': 'req-15ad09aa-644b-46d6-8b86-a687702b819f'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:05.392533 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6d217e15-88bb-4c56-a43d-200f95fcac2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ca2e554196e46c4ad9100622fed470c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:05.400104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-15ad09aa-644b-46d6-8b86-a687702b819f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be918a9e63cb43928a6ee4206890f323', 'request_id': 'req-15ad09aa-644b-46d6-8b86-a687702b819f'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:05.403466 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6d217e15-88bb-4c56-a43d-200f95fcac2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ca2e554196e46c4ad9100622fed470c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:05.404578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6d217e15-88bb-4c56-a43d-200f95fcac2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ca2e554196e46c4ad9100622fed470c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:05.411775 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-6d217e15-88bb-4c56-a43d-200f95fcac2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ca2e554196e46c4ad9100622fed470c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:05.415386 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-15ad09aa-644b-46d6-8b86-a687702b819f admin admin] The token cache is being invalidate because project be918a9e63cb43928a6ee4206890f323 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:14:05.415825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-15ad09aa-644b-46d6-8b86-a687702b819f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:05.416722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-15ad09aa-644b-46d6-8b86-a687702b819f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:05.418520 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1115/2224] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:14:05 2022] DELETE /identity/v3/projects/be918a9e63cb43928a6ee4206890f323 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:14:05.422759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1110/2225] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:14:05 2022] DELETE /identity/v3/users/0ca2e554196e46c4ad9100622fed470c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:05.423877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1f13f-aac7-44a8-8925-3a4dae901e76 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:05.424163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d366e1-f246-4a71-bfee-6ed6926eebe1 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:05.424458 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d366e1-f246-4a71-bfee-6ed6926eebe1 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:05.424772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d366e1-f246-4a71-bfee-6ed6926eebe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:05.424916 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1f13f-aac7-44a8-8925-3a4dae901e76 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:05.425319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1f13f-aac7-44a8-8925-3a4dae901e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:05.443661 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a9d366e1-f246-4a71-bfee-6ed6926eebe1 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:05.445344 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a9d366e1-f246-4a71-bfee-6ed6926eebe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:05.449528 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-40a1f13f-aac7-44a8-8925-3a4dae901e76 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:05.451729 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-40a1f13f-aac7-44a8-8925-3a4dae901e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:05.513824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1111/2226] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:05.523881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d366e1-f246-4a71-bfee-6ed6926eebe1 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:05.527497 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a9d366e1-f246-4a71-bfee-6ed6926eebe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:05.533032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1116/2227] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:05.541422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd22ef-94ec-440c-a819-645242c7591d None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:05.541422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd22ef-94ec-440c-a819-645242c7591d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:05.541750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd22ef-94ec-440c-a819-645242c7591d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:05.568773 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-58dd22ef-94ec-440c-a819-645242c7591d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:05.570380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-58dd22ef-94ec-440c-a819-645242c7591d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:05.579671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1117/2228] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:05.584130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd22ef-94ec-440c-a819-645242c7591d None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:05.595102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd22ef-94ec-440c-a819-645242c7591d None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:05.600753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d366e1-f246-4a71-bfee-6ed6926eebe1 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:05.601561 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732bfcb2-02fe-475d-be9c-389547af03e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:05.602228 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bfcb2-02fe-475d-be9c-389547af03e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:05.602520 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bfcb2-02fe-475d-be9c-389547af03e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:05.602822 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bfcb2-02fe-475d-be9c-389547af03e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:05.604108 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732bfcb2-02fe-475d-be9c-389547af03e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:14:05.604429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732bfcb2-02fe-475d-be9c-389547af03e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:05.605872 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef04c5d-8a78-49d8-a35a-72be57dc4b68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:05.606134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732bfcb2-02fe-475d-be9c-389547af03e2 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:05.606662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef04c5d-8a78-49d8-a35a-72be57dc4b68 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:05.607037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef04c5d-8a78-49d8-a35a-72be57dc4b68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:05.607423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef04c5d-8a78-49d8-a35a-72be57dc4b68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:05.618188 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef04c5d-8a78-49d8-a35a-72be57dc4b68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:05.619691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef04c5d-8a78-49d8-a35a-72be57dc4b68 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:05.622762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1118/2229] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:14:05.627201 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732bfcb2-02fe-475d-be9c-389547af03e2 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:05.630533 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1112/2230] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:14:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:14:05.639202 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732bfcb2-02fe-475d-be9c-389547af03e2 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:05.644712 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615f271-52b4-455f-99cf-3575805b4b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:05.645572 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615f271-52b4-455f-99cf-3575805b4b18 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:05.645866 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615f271-52b4-455f-99cf-3575805b4b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:05.646152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615f271-52b4-455f-99cf-3575805b4b18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:05.646993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615f271-52b4-455f-99cf-3575805b4b18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:05.649890 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615f271-52b4-455f-99cf-3575805b4b18 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:05.683595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1119/2231] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:14:05 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:14:05.685046 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef04c5d-8a78-49d8-a35a-72be57dc4b68 service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:05.696813 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef04c5d-8a78-49d8-a35a-72be57dc4b68 service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:05.810175 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7382bb3-6220-43fa-9471-08277584160c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:05.810897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7382bb3-6220-43fa-9471-08277584160c admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:05.811183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7382bb3-6220-43fa-9471-08277584160c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:05.811461 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7382bb3-6220-43fa-9471-08277584160c admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:05.813029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7382bb3-6220-43fa-9471-08277584160c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:05.815369 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7382bb3-6220-43fa-9471-08277584160c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:05.843951 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1113/2232] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:14:05 2022] POST /identity/v3/users => generated 282 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:14:05.845764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615f271-52b4-455f-99cf-3575805b4b18 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:05.858828 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615f271-52b4-455f-99cf-3575805b4b18 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:05.864217 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ac6d0-af5e-4be5-8a68-14d79facf978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:05.864900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ac6d0-af5e-4be5-8a68-14d79facf978 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:05.865570 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ac6d0-af5e-4be5-8a68-14d79facf978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:05.865570 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ac6d0-af5e-4be5-8a68-14d79facf978 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:05.866252 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ac6d0-af5e-4be5-8a68-14d79facf978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:05.869611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ac6d0-af5e-4be5-8a68-14d79facf978 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:05.883985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1120/2233] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:14:05.888073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7382bb3-6220-43fa-9471-08277584160c admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:05.899061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7382bb3-6220-43fa-9471-08277584160c admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:05.904227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6eee91-37cb-4c0e-88a9-bd37aa9b442a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:05.904853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6eee91-37cb-4c0e-88a9-bd37aa9b442a admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:05.905134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6eee91-37cb-4c0e-88a9-bd37aa9b442a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:05.905578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6eee91-37cb-4c0e-88a9-bd37aa9b442a admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:05.906323 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6eee91-37cb-4c0e-88a9-bd37aa9b442a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:05.907966 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6eee91-37cb-4c0e-88a9-bd37aa9b442a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:05.910744 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:05.926435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ac6d0-af5e-4be5-8a68-14d79facf978 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:05.933370 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1114/2234] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:14:05.938871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ac6d0-af5e-4be5-8a68-14d79facf978 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:05.944255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a1613-2702-41c0-87a6-8c5f14479870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:05.945858 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a1613-2702-41c0-87a6-8c5f14479870 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:05.946120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a1613-2702-41c0-87a6-8c5f14479870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:05.946657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a1613-2702-41c0-87a6-8c5f14479870 admin admin] PATH_INFO: `/v3/projects/a8bac21ce60f4be39defd7fc6611a395/users/56e6f115b3f24109a2589d6064f0c3f7/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:05.958715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6eee91-37cb-4c0e-88a9-bd37aa9b442a admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:05.961091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3f6eee91-37cb-4c0e-88a9-bd37aa9b442a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:05.977794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a1613-2702-41c0-87a6-8c5f14479870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8bac21ce60f4be39defd7fc6611a395, user_id=56e6f115b3f24109a2589d6064f0c3f7, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:05.980994 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a1613-2702-41c0-87a6-8c5f14479870 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:05.983538 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:05.988172 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6eee91-37cb-4c0e-88a9-bd37aa9b442a admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:05.995692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def507e-efa9-491f-8c19-7e5ad16b9ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:05.997430 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def507e-efa9-491f-8c19-7e5ad16b9ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:05.997697 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def507e-efa9-491f-8c19-7e5ad16b9ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:05.997987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def507e-efa9-491f-8c19-7e5ad16b9ce0 admin admin] PATH_INFO: `/v3/projects/19570f5948f2444596e88411c6996ea1` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:05.999523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def507e-efa9-491f-8c19-7e5ad16b9ce0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19570f5948f2444596e88411c6996ea1)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:06.001771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def507e-efa9-491f-8c19-7e5ad16b9ce0 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:06.004129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1121/2235] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:14:05 2022] PUT /identity/v3/projects/a8bac21ce60f4be39defd7fc6611a395/users/56e6f115b3f24109a2589d6064f0c3f7/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:14:06.015546 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5832a0-b01e-4e3c-8f57-d5c1324d398e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:06.015864 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5832a0-b01e-4e3c-8f57-d5c1324d398e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:06.016150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5832a0-b01e-4e3c-8f57-d5c1324d398e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:06.045801 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-8a5832a0-b01e-4e3c-8f57-d5c1324d398e None None] MFA Rules not processed for user `56e6f115b3f24109a2589d6064f0c3f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:06.047634 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8a5832a0-b01e-4e3c-8f57-d5c1324d398e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:06.067818 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1def507e-efa9-491f-8c19-7e5ad16b9ce0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19570f5948f2444596e88411c6996ea1', 'request_id': 'req-1def507e-efa9-491f-8c19-7e5ad16b9ce0'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:06.075905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1def507e-efa9-491f-8c19-7e5ad16b9ce0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19570f5948f2444596e88411c6996ea1', 'request_id': 'req-1def507e-efa9-491f-8c19-7e5ad16b9ce0'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:06.085347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1def507e-efa9-491f-8c19-7e5ad16b9ce0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19570f5948f2444596e88411c6996ea1', 'request_id': 'req-1def507e-efa9-491f-8c19-7e5ad16b9ce0'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:06.098145 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1def507e-efa9-491f-8c19-7e5ad16b9ce0 admin admin] The token cache is being invalidate because project 19570f5948f2444596e88411c6996ea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:14:06.098440 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1def507e-efa9-491f-8c19-7e5ad16b9ce0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:06.099449 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1def507e-efa9-491f-8c19-7e5ad16b9ce0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:06.101443 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1115/2236] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:14:05 2022] DELETE /identity/v3/projects/19570f5948f2444596e88411c6996ea1 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:06.105604 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def507e-efa9-491f-8c19-7e5ad16b9ce0 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:06.107450 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-1def507e-efa9-491f-8c19-7e5ad16b9ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:06.134990 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1122/2237] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:06 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:06.143100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5832a0-b01e-4e3c-8f57-d5c1324d398e None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:06.145462 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8a5832a0-b01e-4e3c-8f57-d5c1324d398e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:06.188608 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def507e-efa9-491f-8c19-7e5ad16b9ce0 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:06.193547 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479deabc-b5fe-4440-8964-af385d46e59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:06.194500 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479deabc-b5fe-4440-8964-af385d46e59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:06.195446 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479deabc-b5fe-4440-8964-af385d46e59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:06.195798 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479deabc-b5fe-4440-8964-af385d46e59e admin admin] PATH_INFO: `/v3/users/d8544eaefee545baaf05b6af9bbb83f8` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:06.197236 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479deabc-b5fe-4440-8964-af385d46e59e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8544eaefee545baaf05b6af9bbb83f8)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:06.199605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479deabc-b5fe-4440-8964-af385d46e59e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:06.209273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5832a0-b01e-4e3c-8f57-d5c1324d398e None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:06.255066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-479deabc-b5fe-4440-8964-af385d46e59e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8544eaefee545baaf05b6af9bbb83f8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:06.261435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bac8a-5032-442b-af93-8481ffc084d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:06.262098 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bac8a-5032-442b-af93-8481ffc084d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:06.262386 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bac8a-5032-442b-af93-8481ffc084d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:06.262668 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bac8a-5032-442b-af93-8481ffc084d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:06.283131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-479deabc-b5fe-4440-8964-af385d46e59e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8544eaefee545baaf05b6af9bbb83f8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:06.299057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-479deabc-b5fe-4440-8964-af385d46e59e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8544eaefee545baaf05b6af9bbb83f8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:06.299847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-479deabc-b5fe-4440-8964-af385d46e59e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8544eaefee545baaf05b6af9bbb83f8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:06.313789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-479deabc-b5fe-4440-8964-af385d46e59e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8544eaefee545baaf05b6af9bbb83f8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:06.320872 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:06.325510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1116/2238] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:14:06 2022] DELETE /identity/v3/users/d8544eaefee545baaf05b6af9bbb83f8 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:14:06.350494 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:06.355942 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bac8a-5032-442b-af93-8481ffc084d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:06.357953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bac8a-5032-442b-af93-8481ffc084d2 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:06.383049 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1123/2239] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:14:06 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:14:06.752562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479deabc-b5fe-4440-8964-af385d46e59e admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:06.754833 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-479deabc-b5fe-4440-8964-af385d46e59e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:06.887052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479deabc-b5fe-4440-8964-af385d46e59e admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:06.892412 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b8f5-a625-4166-ab95-168e6ee6be74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:06.893301 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b8f5-a625-4166-ab95-168e6ee6be74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:06.893590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b8f5-a625-4166-ab95-168e6ee6be74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:06.893874 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b8f5-a625-4166-ab95-168e6ee6be74 admin admin] PATH_INFO: `/v3/projects/ce1b011c885b4d35af18a4c1587c01c6` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:06.895133 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69b8f5-a625-4166-ab95-168e6ee6be74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce1b011c885b4d35af18a4c1587c01c6)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:06.897090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69b8f5-a625-4166-ab95-168e6ee6be74 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:06.947348 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0e69b8f5-a625-4166-ab95-168e6ee6be74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce1b011c885b4d35af18a4c1587c01c6', 'request_id': 'req-0e69b8f5-a625-4166-ab95-168e6ee6be74'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:06.953490 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0e69b8f5-a625-4166-ab95-168e6ee6be74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce1b011c885b4d35af18a4c1587c01c6', 'request_id': 'req-0e69b8f5-a625-4166-ab95-168e6ee6be74'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:06.961769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0e69b8f5-a625-4166-ab95-168e6ee6be74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce1b011c885b4d35af18a4c1587c01c6', 'request_id': 'req-0e69b8f5-a625-4166-ab95-168e6ee6be74'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:06.973105 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0e69b8f5-a625-4166-ab95-168e6ee6be74 admin admin] The token cache is being invalidate because project ce1b011c885b4d35af18a4c1587c01c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:14:06.973420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0e69b8f5-a625-4166-ab95-168e6ee6be74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:06.974061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0e69b8f5-a625-4166-ab95-168e6ee6be74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:06.976453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1117/2240] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:14:06 2022] DELETE /identity/v3/projects/ce1b011c885b4d35af18a4c1587c01c6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:06.980371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bac8a-5032-442b-af93-8481ffc084d2 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:06.993458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bac8a-5032-442b-af93-8481ffc084d2 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:07.057020 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5a834-e504-43b8-9cd5-f75185dadff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:07.058011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5a834-e504-43b8-9cd5-f75185dadff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:07.058949 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5a834-e504-43b8-9cd5-f75185dadff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:07.058949 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5a834-e504-43b8-9cd5-f75185dadff0 admin admin] PATH_INFO: `/v3/users/1bc4afc79c084138a85a26c7114ab7c7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:07.060056 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5a834-e504-43b8-9cd5-f75185dadff0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bc4afc79c084138a85a26c7114ab7c7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:07.062268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5a834-e504-43b8-9cd5-f75185dadff0 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:07.139154 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-96e5a834-e504-43b8-9cd5-f75185dadff0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bc4afc79c084138a85a26c7114ab7c7'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:07.155726 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-96e5a834-e504-43b8-9cd5-f75185dadff0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bc4afc79c084138a85a26c7114ab7c7'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:07.169816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-96e5a834-e504-43b8-9cd5-f75185dadff0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bc4afc79c084138a85a26c7114ab7c7'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:07.171628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-96e5a834-e504-43b8-9cd5-f75185dadff0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bc4afc79c084138a85a26c7114ab7c7'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:07.183693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-96e5a834-e504-43b8-9cd5-f75185dadff0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bc4afc79c084138a85a26c7114ab7c7'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:07.192569 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1124/2241] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:14:06 2022] DELETE /identity/v3/users/1bc4afc79c084138a85a26c7114ab7c7 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:07.544053 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b8f5-a625-4166-ab95-168e6ee6be74 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:07.545529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0e69b8f5-a625-4166-ab95-168e6ee6be74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:07.568575 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b8f5-a625-4166-ab95-168e6ee6be74 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:07.668875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4913df2c-d5be-4a46-8a4d-a3718c6573f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:07.669964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913df2c-d5be-4a46-8a4d-a3718c6573f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:07.676574 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913df2c-d5be-4a46-8a4d-a3718c6573f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:07.676811 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913df2c-d5be-4a46-8a4d-a3718c6573f0 admin admin] PATH_INFO: `/v3/projects/a488f82ad49f42b8af0d9b664a96140a` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:07.677989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4913df2c-d5be-4a46-8a4d-a3718c6573f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a488f82ad49f42b8af0d9b664a96140a)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:07.681014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4913df2c-d5be-4a46-8a4d-a3718c6573f0 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:07.730317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:07.749575 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4913df2c-d5be-4a46-8a4d-a3718c6573f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a488f82ad49f42b8af0d9b664a96140a', 'request_id': 'req-4913df2c-d5be-4a46-8a4d-a3718c6573f0'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:07.756193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4913df2c-d5be-4a46-8a4d-a3718c6573f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a488f82ad49f42b8af0d9b664a96140a', 'request_id': 'req-4913df2c-d5be-4a46-8a4d-a3718c6573f0'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:07.766877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4913df2c-d5be-4a46-8a4d-a3718c6573f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a488f82ad49f42b8af0d9b664a96140a', 'request_id': 'req-4913df2c-d5be-4a46-8a4d-a3718c6573f0'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:07.780977 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4913df2c-d5be-4a46-8a4d-a3718c6573f0 admin admin] The token cache is being invalidate because project a488f82ad49f42b8af0d9b664a96140a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:14:07.781317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4913df2c-d5be-4a46-8a4d-a3718c6573f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:07.781978 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4913df2c-d5be-4a46-8a4d-a3718c6573f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:07.783864 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1118/2242] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:14:07 2022] DELETE /identity/v3/projects/a488f82ad49f42b8af0d9b664a96140a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:07.788912 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f98fc-9b4e-438c-978c-ff4bf94650a9 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:07.789673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f98fc-9b4e-438c-978c-ff4bf94650a9 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:07.789673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f98fc-9b4e-438c-978c-ff4bf94650a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:07.808826 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-631f98fc-9b4e-438c-978c-ff4bf94650a9 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:07.810077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-631f98fc-9b4e-438c-978c-ff4bf94650a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:07.885734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1125/2243] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:07.893775 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2b337-26b6-4401-ba9a-92c2aa9194f0 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:07.894070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2b337-26b6-4401-ba9a-92c2aa9194f0 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:07.894360 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2b337-26b6-4401-ba9a-92c2aa9194f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:07.915945 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-e0a2b337-26b6-4401-ba9a-92c2aa9194f0 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:07.917193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e0a2b337-26b6-4401-ba9a-92c2aa9194f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:07.929535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1119/2244] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:07.930104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f98fc-9b4e-438c-978c-ff4bf94650a9 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:07.940763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f98fc-9b4e-438c-978c-ff4bf94650a9 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:07.946705 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8c1f1-685c-4b10-aba1-a7ca91a2f3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:07.949051 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8c1f1-685c-4b10-aba1-a7ca91a2f3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:07.949051 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8c1f1-685c-4b10-aba1-a7ca91a2f3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:07.949051 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8c1f1-685c-4b10-aba1-a7ca91a2f3c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:07.949321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8c1f1-685c-4b10-aba1-a7ca91a2f3c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:14:07.949597 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8c1f1-685c-4b10-aba1-a7ca91a2f3c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:07.951340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8c1f1-685c-4b10-aba1-a7ca91a2f3c1 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:07.963931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:07.969341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1126/2245] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:14:07.975056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2b337-26b6-4401-ba9a-92c2aa9194f0 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:07.992092 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2b337-26b6-4401-ba9a-92c2aa9194f0 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:07.997332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccca4cc-a47d-4df1-8387-d53f10688c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:07.998111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccca4cc-a47d-4df1-8387-d53f10688c38 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:07.998623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccca4cc-a47d-4df1-8387-d53f10688c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:07.999025 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccca4cc-a47d-4df1-8387-d53f10688c38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:07.999802 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccca4cc-a47d-4df1-8387-d53f10688c38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:08.001870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccca4cc-a47d-4df1-8387-d53f10688c38 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:08.066719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1120/2246] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:14:07 2022] POST /identity/v3/projects => generated 399 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:14:08.069509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8c1f1-685c-4b10-aba1-a7ca91a2f3c1 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:08.084316 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8c1f1-685c-4b10-aba1-a7ca91a2f3c1 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:08.223321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad0251-0e22-465c-b068-d44b4c6db155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:08.229490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad0251-0e22-465c-b068-d44b4c6db155 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:08.229664 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad0251-0e22-465c-b068-d44b4c6db155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:08.229936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad0251-0e22-465c-b068-d44b4c6db155 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:08.230668 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad0251-0e22-465c-b068-d44b4c6db155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:08.232697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad0251-0e22-465c-b068-d44b4c6db155 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:08.283486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1127/2247] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:14:08 2022] POST /identity/v3/users => generated 300 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:14:08.287519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccca4cc-a47d-4df1-8387-d53f10688c38 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:08.316944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccca4cc-a47d-4df1-8387-d53f10688c38 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:08.324923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd062a-71f2-4394-9acd-e21f8cb59c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:08.326160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd062a-71f2-4394-9acd-e21f8cb59c6c admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:08.326592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd062a-71f2-4394-9acd-e21f8cb59c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:08.327076 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd062a-71f2-4394-9acd-e21f8cb59c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:08.327827 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd062a-71f2-4394-9acd-e21f8cb59c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:08.329444 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd062a-71f2-4394-9acd-e21f8cb59c6c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:08.367403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad0251-0e22-465c-b068-d44b4c6db155 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:08.368136 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1121/2248] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:14:08.389135 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad0251-0e22-465c-b068-d44b4c6db155 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:08.407933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426c9e6-4934-408e-b6dc-614631d9e116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:08.408825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426c9e6-4934-408e-b6dc-614631d9e116 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:08.409180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426c9e6-4934-408e-b6dc-614631d9e116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:08.409472 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426c9e6-4934-408e-b6dc-614631d9e116 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:08.410230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426c9e6-4934-408e-b6dc-614631d9e116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:08.411891 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426c9e6-4934-408e-b6dc-614631d9e116 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:08.448751 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1128/2249] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:14:08.453310 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd062a-71f2-4394-9acd-e21f8cb59c6c admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:08.468030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd062a-71f2-4394-9acd-e21f8cb59c6c admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:08.492419 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491f9a0-a548-48d5-8f40-62b938ceccd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:08.493165 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491f9a0-a548-48d5-8f40-62b938ceccd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:08.493594 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491f9a0-a548-48d5-8f40-62b938ceccd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:08.497382 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491f9a0-a548-48d5-8f40-62b938ceccd4 admin admin] PATH_INFO: `/v3/projects/c3294c0ff4724c4fa6cd9b44e167a81f/users/32c816b1ad684f8790a7e9687f9e2e43/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:08.542726 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e491f9a0-a548-48d5-8f40-62b938ceccd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3294c0ff4724c4fa6cd9b44e167a81f, user_id=32c816b1ad684f8790a7e9687f9e2e43, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:08.545605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e491f9a0-a548-48d5-8f40-62b938ceccd4 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:08.573148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1122/2250] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:14:08 2022] PUT /identity/v3/projects/c3294c0ff4724c4fa6cd9b44e167a81f/users/32c816b1ad684f8790a7e9687f9e2e43/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:08.657776 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0faa11-e2db-4e26-a33f-40e90284267d None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:08.658312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0faa11-e2db-4e26-a33f-40e90284267d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:08.658910 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0faa11-e2db-4e26-a33f-40e90284267d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:08.732650 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-8b0faa11-e2db-4e26-a33f-40e90284267d None None] MFA Rules not processed for user `32c816b1ad684f8790a7e9687f9e2e43`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:08.735471 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8b0faa11-e2db-4e26-a33f-40e90284267d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:08.894797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1129/2251] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:14:08.908056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491f9a0-a548-48d5-8f40-62b938ceccd4 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:08.910122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e491f9a0-a548-48d5-8f40-62b938ceccd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:09.004830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491f9a0-a548-48d5-8f40-62b938ceccd4 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:09.013171 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e337c5-32f7-4796-a9f9-25206d6a514e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:09.014126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e337c5-32f7-4796-a9f9-25206d6a514e service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:09.014822 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e337c5-32f7-4796-a9f9-25206d6a514e service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:09.015140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e337c5-32f7-4796-a9f9-25206d6a514e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:09.025759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e337c5-32f7-4796-a9f9-25206d6a514e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:09.027321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e337c5-32f7-4796-a9f9-25206d6a514e service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:09.038063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1123/2252] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:14:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:14:10.084569 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0faa11-e2db-4e26-a33f-40e90284267d None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:10.086624 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8b0faa11-e2db-4e26-a33f-40e90284267d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:10.245453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0faa11-e2db-4e26-a33f-40e90284267d None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:10.251167 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a3a5c1-998c-4477-92f1-8455a48a9bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:10.252095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3a5c1-998c-4477-92f1-8455a48a9bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:10.252418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3a5c1-998c-4477-92f1-8455a48a9bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:10.253495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3a5c1-998c-4477-92f1-8455a48a9bca admin admin] PATH_INFO: `/v3/users/a24a58daa8ed4d77a649a670cc358638` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:10.264078 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a3a5c1-998c-4477-92f1-8455a48a9bca admin admin] RBAC: Authorizing `identity:delete_user(user_id=a24a58daa8ed4d77a649a670cc358638)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:10.266425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a3a5c1-998c-4477-92f1-8455a48a9bca admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:10.282405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:10.337813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-39a3a5c1-998c-4477-92f1-8455a48a9bca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a24a58daa8ed4d77a649a670cc358638'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:10.350178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-39a3a5c1-998c-4477-92f1-8455a48a9bca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a24a58daa8ed4d77a649a670cc358638'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:10.357682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-39a3a5c1-998c-4477-92f1-8455a48a9bca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a24a58daa8ed4d77a649a670cc358638'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:10.358724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-39a3a5c1-998c-4477-92f1-8455a48a9bca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a24a58daa8ed4d77a649a670cc358638'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:10.369268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-39a3a5c1-998c-4477-92f1-8455a48a9bca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a24a58daa8ed4d77a649a670cc358638'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:10.375568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1130/2253] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:14:10 2022] DELETE /identity/v3/users/a24a58daa8ed4d77a649a670cc358638 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:10.572799 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e337c5-32f7-4796-a9f9-25206d6a514e service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:10.591382 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e337c5-32f7-4796-a9f9-25206d6a514e service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:10.652017 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf01f8a-f6f9-41b2-b4f0-3d5f6c3bc3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:10.652890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf01f8a-f6f9-41b2-b4f0-3d5f6c3bc3a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:10.653176 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf01f8a-f6f9-41b2-b4f0-3d5f6c3bc3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:10.653457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf01f8a-f6f9-41b2-b4f0-3d5f6c3bc3a5 admin admin] PATH_INFO: `/v3/projects/e286919dacfc4db1a83308683c2a23ba` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:10.654640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf01f8a-f6f9-41b2-b4f0-3d5f6c3bc3a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e286919dacfc4db1a83308683c2a23ba)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:10.656645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf01f8a-f6f9-41b2-b4f0-3d5f6c3bc3a5 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:10.658287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:10.708262 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-faf01f8a-f6f9-41b2-b4f0-3d5f6c3bc3a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e286919dacfc4db1a83308683c2a23ba', 'request_id': 'req-faf01f8a-f6f9-41b2-b4f0-3d5f6c3bc3a5'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:10.714781 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-faf01f8a-f6f9-41b2-b4f0-3d5f6c3bc3a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e286919dacfc4db1a83308683c2a23ba', 'request_id': 'req-faf01f8a-f6f9-41b2-b4f0-3d5f6c3bc3a5'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:10.722642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-faf01f8a-f6f9-41b2-b4f0-3d5f6c3bc3a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e286919dacfc4db1a83308683c2a23ba', 'request_id': 'req-faf01f8a-f6f9-41b2-b4f0-3d5f6c3bc3a5'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:10.734032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-faf01f8a-f6f9-41b2-b4f0-3d5f6c3bc3a5 admin admin] The token cache is being invalidate because project e286919dacfc4db1a83308683c2a23ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:14:10.734330 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-faf01f8a-f6f9-41b2-b4f0-3d5f6c3bc3a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:10.734997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-faf01f8a-f6f9-41b2-b4f0-3d5f6c3bc3a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:10.736534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1124/2254] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:14:10 2022] DELETE /identity/v3/projects/e286919dacfc4db1a83308683c2a23ba => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:10.742243 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df3fdc-533a-4ad8-a924-60afd425c10e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:10.742631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df3fdc-533a-4ad8-a924-60afd425c10e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:10.742983 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df3fdc-533a-4ad8-a924-60afd425c10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:10.761366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-03df3fdc-533a-4ad8-a924-60afd425c10e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:10.762659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-03df3fdc-533a-4ad8-a924-60afd425c10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:10.825187 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1131/2255] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:10.833692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d219b-1b2f-448c-b5d5-b831ec3c2464 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:10.834050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d219b-1b2f-448c-b5d5-b831ec3c2464 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:10.834399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d219b-1b2f-448c-b5d5-b831ec3c2464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:10.853255 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-8d6d219b-1b2f-448c-b5d5-b831ec3c2464 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:10.854610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8d6d219b-1b2f-448c-b5d5-b831ec3c2464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:10.863438 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1125/2256] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:10.868512 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df3fdc-533a-4ad8-a924-60afd425c10e None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:10.878971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df3fdc-533a-4ad8-a924-60afd425c10e None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:10.885008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731786d-8b27-4425-b6e8-271f0a82060b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:10.885823 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731786d-8b27-4425-b6e8-271f0a82060b admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:10.886245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731786d-8b27-4425-b6e8-271f0a82060b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:10.886532 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731786d-8b27-4425-b6e8-271f0a82060b admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:10.887298 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731786d-8b27-4425-b6e8-271f0a82060b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:14:10.887602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731786d-8b27-4425-b6e8-271f0a82060b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:10.889221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731786d-8b27-4425-b6e8-271f0a82060b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:10.901174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1132/2257] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:14:10.905621 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d219b-1b2f-448c-b5d5-b831ec3c2464 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:10.916259 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d219b-1b2f-448c-b5d5-b831ec3c2464 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:10.921952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86794e-2647-4fb7-ab1d-0d8eab35493d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:10.922630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86794e-2647-4fb7-ab1d-0d8eab35493d admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:10.922949 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86794e-2647-4fb7-ab1d-0d8eab35493d admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:10.923232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86794e-2647-4fb7-ab1d-0d8eab35493d admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:10.924415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86794e-2647-4fb7-ab1d-0d8eab35493d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:10.926727 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d86794e-2647-4fb7-ab1d-0d8eab35493d admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:10.961591 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1126/2258] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:14:10 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:14:10.966089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731786d-8b27-4425-b6e8-271f0a82060b admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:10.978171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731786d-8b27-4425-b6e8-271f0a82060b admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:11.039113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2331ac9-f7c7-447c-b56f-ab68e5921113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:11.039864 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2331ac9-f7c7-447c-b56f-ab68e5921113 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:11.040087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2331ac9-f7c7-447c-b56f-ab68e5921113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:11.040372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2331ac9-f7c7-447c-b56f-ab68e5921113 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:11.041079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2331ac9-f7c7-447c-b56f-ab68e5921113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:11.043014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2331ac9-f7c7-447c-b56f-ab68e5921113 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:11.072572 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1133/2259] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:14:10 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:14:11.074556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86794e-2647-4fb7-ab1d-0d8eab35493d admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:11.085191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86794e-2647-4fb7-ab1d-0d8eab35493d admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:11.090262 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811dc8f3-40cb-47ad-9ae1-be9e3ff06e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:11.090969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811dc8f3-40cb-47ad-9ae1-be9e3ff06e6d admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:11.091255 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811dc8f3-40cb-47ad-9ae1-be9e3ff06e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:11.091533 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811dc8f3-40cb-47ad-9ae1-be9e3ff06e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:11.092389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811dc8f3-40cb-47ad-9ae1-be9e3ff06e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:11.093984 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811dc8f3-40cb-47ad-9ae1-be9e3ff06e6d admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:11.105411 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1127/2260] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:14:11.109763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2331ac9-f7c7-447c-b56f-ab68e5921113 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:11.116385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:11.121601 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2331ac9-f7c7-447c-b56f-ab68e5921113 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:11.129771 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd569c5d-6659-43ea-9d14-2eac9d519820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:11.130352 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd569c5d-6659-43ea-9d14-2eac9d519820 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:11.131229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd569c5d-6659-43ea-9d14-2eac9d519820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:11.131418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd569c5d-6659-43ea-9d14-2eac9d519820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:11.132771 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd569c5d-6659-43ea-9d14-2eac9d519820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:11.134633 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd569c5d-6659-43ea-9d14-2eac9d519820 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:11.145590 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1134/2261] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:14:11.150349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811dc8f3-40cb-47ad-9ae1-be9e3ff06e6d admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:11.161275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811dc8f3-40cb-47ad-9ae1-be9e3ff06e6d admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:11.166444 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41e1e4-897e-4ea6-b227-8c5be48a6fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:11.167404 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41e1e4-897e-4ea6-b227-8c5be48a6fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:11.167690 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41e1e4-897e-4ea6-b227-8c5be48a6fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:11.167967 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41e1e4-897e-4ea6-b227-8c5be48a6fc0 admin admin] PATH_INFO: `/v3/projects/6476016c570542d3b00f9ed1507655da/users/0a3ed8bea97e49628311cb45d48ca699/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:11.183726 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41e1e4-897e-4ea6-b227-8c5be48a6fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6476016c570542d3b00f9ed1507655da, user_id=0a3ed8bea97e49628311cb45d48ca699, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:11.186585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41e1e4-897e-4ea6-b227-8c5be48a6fc0 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:11.199881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1128/2262] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:14:11 2022] PUT /identity/v3/projects/6476016c570542d3b00f9ed1507655da/users/0a3ed8bea97e49628311cb45d48ca699/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:11.204862 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee446b-788b-471b-99c4-ed33e663b037 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:11.205159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee446b-788b-471b-99c4-ed33e663b037 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:11.205879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee446b-788b-471b-99c4-ed33e663b037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:11.223471 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-c5ee446b-788b-471b-99c4-ed33e663b037 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:11.224898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c5ee446b-788b-471b-99c4-ed33e663b037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:11.307170 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1135/2263] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:11.314586 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41e1e4-897e-4ea6-b227-8c5be48a6fc0 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:11.320567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ab41e1e4-897e-4ea6-b227-8c5be48a6fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:11.391995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41e1e4-897e-4ea6-b227-8c5be48a6fc0 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:11.396192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fe732-35f6-4c99-9bba-cbb3210692f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:11.396957 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fe732-35f6-4c99-9bba-cbb3210692f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:11.398324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fe732-35f6-4c99-9bba-cbb3210692f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:11.398324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fe732-35f6-4c99-9bba-cbb3210692f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:11.407950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fe732-35f6-4c99-9bba-cbb3210692f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:11.409536 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fe732-35f6-4c99-9bba-cbb3210692f9 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:11.420270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1129/2264] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:14:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:14:14.485210 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee446b-788b-471b-99c4-ed33e663b037 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:14.487997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c5ee446b-788b-471b-99c4-ed33e663b037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:14.513051 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee446b-788b-471b-99c4-ed33e663b037 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:14.530552 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23aa37-6a48-41aa-a329-96897e943f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:14.531453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23aa37-6a48-41aa-a329-96897e943f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:14.531743 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23aa37-6a48-41aa-a329-96897e943f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:14.532033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23aa37-6a48-41aa-a329-96897e943f4f admin admin] PATH_INFO: `/v3/users/0d3c41b927a34022b85d23be6fcbbd9a` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:14.545048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23aa37-6a48-41aa-a329-96897e943f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d3c41b927a34022b85d23be6fcbbd9a)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:14.547065 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23aa37-6a48-41aa-a329-96897e943f4f admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:14.669020 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-4e23aa37-6a48-41aa-a329-96897e943f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d3c41b927a34022b85d23be6fcbbd9a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:14.692284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-4e23aa37-6a48-41aa-a329-96897e943f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d3c41b927a34022b85d23be6fcbbd9a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:14.705621 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-4e23aa37-6a48-41aa-a329-96897e943f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d3c41b927a34022b85d23be6fcbbd9a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:14.706800 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-4e23aa37-6a48-41aa-a329-96897e943f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d3c41b927a34022b85d23be6fcbbd9a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:14.723321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-4e23aa37-6a48-41aa-a329-96897e943f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d3c41b927a34022b85d23be6fcbbd9a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:14.733217 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1136/2265] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:14:14 2022] DELETE /identity/v3/users/0d3c41b927a34022b85d23be6fcbbd9a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:14.737763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8a35c-5161-409d-9d12-7cc2cb3a8004 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:14.738827 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8a35c-5161-409d-9d12-7cc2cb3a8004 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:14.739284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8a35c-5161-409d-9d12-7cc2cb3a8004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:14.763398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-c2f8a35c-5161-409d-9d12-7cc2cb3a8004 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:14.765360 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c2f8a35c-5161-409d-9d12-7cc2cb3a8004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:14.799837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:14.852402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1130/2266] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:14.864960 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23aa37-6a48-41aa-a329-96897e943f4f admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:14.867073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-4e23aa37-6a48-41aa-a329-96897e943f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:14.984864 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23aa37-6a48-41aa-a329-96897e943f4f admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:14.989868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c708a-3a3d-43f8-83bb-fe6b26e36bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:14.990531 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c708a-3a3d-43f8-83bb-fe6b26e36bbb service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:14.991880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c708a-3a3d-43f8-83bb-fe6b26e36bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:14.992262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c708a-3a3d-43f8-83bb-fe6b26e36bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:15.002083 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c708a-3a3d-43f8-83bb-fe6b26e36bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:15.003758 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c708a-3a3d-43f8-83bb-fe6b26e36bbb service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:15.020602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1137/2267] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:14:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:14:15.398169 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8a35c-5161-409d-9d12-7cc2cb3a8004 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:15.401536 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c2f8a35c-5161-409d-9d12-7cc2cb3a8004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:15.422587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8a35c-5161-409d-9d12-7cc2cb3a8004 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:15.440680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ce82c-dafc-4024-993a-f365c6db58f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:15.441753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ce82c-dafc-4024-993a-f365c6db58f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:15.442074 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ce82c-dafc-4024-993a-f365c6db58f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:15.442361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ce82c-dafc-4024-993a-f365c6db58f1 admin admin] PATH_INFO: `/v3/projects/048a306747474af98da675803fb6d44c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:15.444101 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ce82c-dafc-4024-993a-f365c6db58f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=048a306747474af98da675803fb6d44c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:15.449646 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ce82c-dafc-4024-993a-f365c6db58f1 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:15.524984 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4f2ce82c-dafc-4024-993a-f365c6db58f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '048a306747474af98da675803fb6d44c', 'request_id': 'req-4f2ce82c-dafc-4024-993a-f365c6db58f1'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:15.533436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4f2ce82c-dafc-4024-993a-f365c6db58f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '048a306747474af98da675803fb6d44c', 'request_id': 'req-4f2ce82c-dafc-4024-993a-f365c6db58f1'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:15.546951 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4f2ce82c-dafc-4024-993a-f365c6db58f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '048a306747474af98da675803fb6d44c', 'request_id': 'req-4f2ce82c-dafc-4024-993a-f365c6db58f1'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:15.630398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4f2ce82c-dafc-4024-993a-f365c6db58f1 admin admin] The token cache is being invalidate because project 048a306747474af98da675803fb6d44c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:14:15.630896 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4f2ce82c-dafc-4024-993a-f365c6db58f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:15.634085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4f2ce82c-dafc-4024-993a-f365c6db58f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:15.641703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1131/2268] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:14:15 2022] DELETE /identity/v3/projects/048a306747474af98da675803fb6d44c => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:16.581999 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7bd5a-1294-475c-a1e2-a7e10681e614 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:16.582439 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7bd5a-1294-475c-a1e2-a7e10681e614 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:16.582730 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7bd5a-1294-475c-a1e2-a7e10681e614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:16.602942 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ce82c-dafc-4024-993a-f365c6db58f1 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:16.604412 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4f2ce82c-dafc-4024-993a-f365c6db58f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:16.614877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:16.629767 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-7fa7bd5a-1294-475c-a1e2-a7e10681e614 None None] MFA Rules not processed for user `0a3ed8bea97e49628311cb45d48ca699`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:16.634714 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7fa7bd5a-1294-475c-a1e2-a7e10681e614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:16.755742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ce82c-dafc-4024-993a-f365c6db58f1 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:16.764937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1138/2269] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:16.770484 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a200c-fdc0-4ef1-aecd-a21c6823d8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:16.771552 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a200c-fdc0-4ef1-aecd-a21c6823d8dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:16.772167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a200c-fdc0-4ef1-aecd-a21c6823d8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:16.772484 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a200c-fdc0-4ef1-aecd-a21c6823d8dd admin admin] PATH_INFO: `/v3/users/32c816b1ad684f8790a7e9687f9e2e43` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:16.773870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a200c-fdc0-4ef1-aecd-a21c6823d8dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c816b1ad684f8790a7e9687f9e2e43)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:16.776800 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a200c-fdc0-4ef1-aecd-a21c6823d8dd admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:16.780259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7bd5a-1294-475c-a1e2-a7e10681e614 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:16.781704 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7fa7bd5a-1294-475c-a1e2-a7e10681e614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:16.874100 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d39a200c-fdc0-4ef1-aecd-a21c6823d8dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c816b1ad684f8790a7e9687f9e2e43'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:16.888525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:16.906669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d39a200c-fdc0-4ef1-aecd-a21c6823d8dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c816b1ad684f8790a7e9687f9e2e43'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:16.915792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d39a200c-fdc0-4ef1-aecd-a21c6823d8dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c816b1ad684f8790a7e9687f9e2e43'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:16.916686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d39a200c-fdc0-4ef1-aecd-a21c6823d8dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c816b1ad684f8790a7e9687f9e2e43'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:16.925350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d39a200c-fdc0-4ef1-aecd-a21c6823d8dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c816b1ad684f8790a7e9687f9e2e43'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:16.933192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1132/2270] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:14:16 2022] DELETE /identity/v3/users/32c816b1ad684f8790a7e9687f9e2e43 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:16.937785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83b001-e6e8-4cad-b96f-0705dae03c3e None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:16.938148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83b001-e6e8-4cad-b96f-0705dae03c3e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:16.938992 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83b001-e6e8-4cad-b96f-0705dae03c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:16.945120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7bd5a-1294-475c-a1e2-a7e10681e614 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:16.970963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-1e83b001-e6e8-4cad-b96f-0705dae03c3e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:16.972358 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-1e83b001-e6e8-4cad-b96f-0705dae03c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:17.039461 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6db51-90ba-4f7b-8a7b-32ece8772ae7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:17.041010 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6db51-90ba-4f7b-8a7b-32ece8772ae7 service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:17.041679 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6db51-90ba-4f7b-8a7b-32ece8772ae7 service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:17.042263 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6db51-90ba-4f7b-8a7b-32ece8772ae7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:17.044781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-75c6db51-90ba-4f7b-8a7b-32ece8772ae7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:17.052784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1133/2271] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:17.063808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83b001-e6e8-4cad-b96f-0705dae03c3e None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:17.065654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-1e83b001-e6e8-4cad-b96f-0705dae03c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:17.127081 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:17.160569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83b001-e6e8-4cad-b96f-0705dae03c3e None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:17.161886 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6db51-90ba-4f7b-8a7b-32ece8772ae7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:17.164557 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6db51-90ba-4f7b-8a7b-32ece8772ae7 service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:17.165333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553d8f4-c49f-4ea2-8aa0-71d9ecd60e34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:17.166089 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553d8f4-c49f-4ea2-8aa0-71d9ecd60e34 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:17.166471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553d8f4-c49f-4ea2-8aa0-71d9ecd60e34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:17.166886 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553d8f4-c49f-4ea2-8aa0-71d9ecd60e34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:17.176021 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1139/2272] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:14:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:14:17.178021 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553d8f4-c49f-4ea2-8aa0-71d9ecd60e34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:17.179894 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553d8f4-c49f-4ea2-8aa0-71d9ecd60e34 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:17.190885 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1134/2273] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:14:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:14:17.571487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6db51-90ba-4f7b-8a7b-32ece8772ae7 service nova] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:17.572998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-75c6db51-90ba-4f7b-8a7b-32ece8772ae7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:17.599926 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6db51-90ba-4f7b-8a7b-32ece8772ae7 service nova] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:17.614369 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5d6de-30cc-4e6e-9752-53f6c679d633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:17.615642 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5d6de-30cc-4e6e-9752-53f6c679d633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:17.615967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5d6de-30cc-4e6e-9752-53f6c679d633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:17.616800 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5d6de-30cc-4e6e-9752-53f6c679d633 admin admin] PATH_INFO: `/v3/projects/c3294c0ff4724c4fa6cd9b44e167a81f` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:17.618907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5d6de-30cc-4e6e-9752-53f6c679d633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3294c0ff4724c4fa6cd9b44e167a81f)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:17.621222 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5d6de-30cc-4e6e-9752-53f6c679d633 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:17.699428 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-07d5d6de-30cc-4e6e-9752-53f6c679d633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3294c0ff4724c4fa6cd9b44e167a81f', 'request_id': 'req-07d5d6de-30cc-4e6e-9752-53f6c679d633'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:17.707772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-07d5d6de-30cc-4e6e-9752-53f6c679d633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3294c0ff4724c4fa6cd9b44e167a81f', 'request_id': 'req-07d5d6de-30cc-4e6e-9752-53f6c679d633'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:17.718278 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-07d5d6de-30cc-4e6e-9752-53f6c679d633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3294c0ff4724c4fa6cd9b44e167a81f', 'request_id': 'req-07d5d6de-30cc-4e6e-9752-53f6c679d633'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:17.737325 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-07d5d6de-30cc-4e6e-9752-53f6c679d633 admin admin] The token cache is being invalidate because project c3294c0ff4724c4fa6cd9b44e167a81f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:14:17.737680 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-07d5d6de-30cc-4e6e-9752-53f6c679d633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:17.738423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-07d5d6de-30cc-4e6e-9752-53f6c679d633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:17.740948 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1140/2274] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:14:17 2022] DELETE /identity/v3/projects/c3294c0ff4724c4fa6cd9b44e167a81f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:18.863218 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e55b3-28a7-4c2b-95a4-a1c4e69eb438 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:18.863715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e55b3-28a7-4c2b-95a4-a1c4e69eb438 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:18.863803 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e55b3-28a7-4c2b-95a4-a1c4e69eb438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:18.885126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-591e55b3-28a7-4c2b-95a4-a1c4e69eb438 None None] MFA Rules not processed for user `0a3ed8bea97e49628311cb45d48ca699`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:18.886557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-591e55b3-28a7-4c2b-95a4-a1c4e69eb438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:18.960555 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1135/2275] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:27.733717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:28.950214 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=88,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:50.377207 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb09a1-8768-4a31-b33b-e0320507ce07 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:50.378113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb09a1-8768-4a31-b33b-e0320507ce07 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:50.378403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb09a1-8768-4a31-b33b-e0320507ce07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:50.397391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-34cb09a1-8768-4a31-b33b-e0320507ce07 None None] MFA Rules not processed for user `9a020687e95a40f88d521757d8b25619`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:50.398713 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-34cb09a1-8768-4a31-b33b-e0320507ce07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:50.465224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1141/2276] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:55.504352 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e55b3-28a7-4c2b-95a4-a1c4e69eb438 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:55.506374 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-591e55b3-28a7-4c2b-95a4-a1c4e69eb438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:55.578282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e55b3-28a7-4c2b-95a4-a1c4e69eb438 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:55.583430 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2c13a-9c52-4331-8b68-8ca97f444f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:55.588353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2c13a-9c52-4331-8b68-8ca97f444f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:55.588645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2c13a-9c52-4331-8b68-8ca97f444f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:55.588923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2c13a-9c52-4331-8b68-8ca97f444f1f admin admin] PATH_INFO: `/v3/users/56e6f115b3f24109a2589d6064f0c3f7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:55.590173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2c13a-9c52-4331-8b68-8ca97f444f1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=56e6f115b3f24109a2589d6064f0c3f7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:55.592162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2c13a-9c52-4331-8b68-8ca97f444f1f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:55.645405 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-01c2c13a-9c52-4331-8b68-8ca97f444f1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56e6f115b3f24109a2589d6064f0c3f7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:55.661975 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-01c2c13a-9c52-4331-8b68-8ca97f444f1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56e6f115b3f24109a2589d6064f0c3f7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:55.668311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-01c2c13a-9c52-4331-8b68-8ca97f444f1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56e6f115b3f24109a2589d6064f0c3f7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:55.669147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-01c2c13a-9c52-4331-8b68-8ca97f444f1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56e6f115b3f24109a2589d6064f0c3f7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:55.678544 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-01c2c13a-9c52-4331-8b68-8ca97f444f1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56e6f115b3f24109a2589d6064f0c3f7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:55.688329 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1136/2277] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:14:55 2022] DELETE /identity/v3/users/56e6f115b3f24109a2589d6064f0c3f7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:55.690391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d132b5-873e-40f2-9afa-6ade34ca9ff2 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:55.691464 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d132b5-873e-40f2-9afa-6ade34ca9ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:55.691909 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d132b5-873e-40f2-9afa-6ade34ca9ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:55.714129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-27d132b5-873e-40f2-9afa-6ade34ca9ff2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:55.715716 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-27d132b5-873e-40f2-9afa-6ade34ca9ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:55.802940 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1142/2278] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:55.815963 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2c13a-9c52-4331-8b68-8ca97f444f1f admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:55.817622 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-01c2c13a-9c52-4331-8b68-8ca97f444f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:55.818851 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df526ca-6f3b-4cbd-85fa-38255700b4f5 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:55.819146 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df526ca-6f3b-4cbd-85fa-38255700b4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:55.819426 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df526ca-6f3b-4cbd-85fa-38255700b4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:55.838487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-4df526ca-6f3b-4cbd-85fa-38255700b4f5 None None] MFA Rules not processed for user `abdb339c5f5740d6aff4dfd1066f9384`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:55.839893 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-4df526ca-6f3b-4cbd-85fa-38255700b4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:55.883238 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2c13a-9c52-4331-8b68-8ca97f444f1f admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:55.886926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1cffb-0287-416f-bd3d-878950314b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:55.887581 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1cffb-0287-416f-bd3d-878950314b9d service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:55.887867 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1cffb-0287-416f-bd3d-878950314b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:55.888149 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1cffb-0287-416f-bd3d-878950314b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:55.896122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1143/2279] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:55.897488 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d1cffb-0287-416f-bd3d-878950314b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:55.899041 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d1cffb-0287-416f-bd3d-878950314b9d service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:55.909060 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1137/2280] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:14:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:14:56.286272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df526ca-6f3b-4cbd-85fa-38255700b4f5 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:56.287848 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-4df526ca-6f3b-4cbd-85fa-38255700b4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:56.301813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df526ca-6f3b-4cbd-85fa-38255700b4f5 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:56.307458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c49d2d-6adf-4b94-9f2a-e8290fd1f9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:56.308545 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c49d2d-6adf-4b94-9f2a-e8290fd1f9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:56.308827 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c49d2d-6adf-4b94-9f2a-e8290fd1f9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:56.309123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c49d2d-6adf-4b94-9f2a-e8290fd1f9b8 admin admin] PATH_INFO: `/v3/projects/a8bac21ce60f4be39defd7fc6611a395` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:56.310471 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c49d2d-6adf-4b94-9f2a-e8290fd1f9b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8bac21ce60f4be39defd7fc6611a395)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:56.312448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c49d2d-6adf-4b94-9f2a-e8290fd1f9b8 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:56.370867 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:56.373712 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-20c49d2d-6adf-4b94-9f2a-e8290fd1f9b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8bac21ce60f4be39defd7fc6611a395', 'request_id': 'req-20c49d2d-6adf-4b94-9f2a-e8290fd1f9b8'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:56.381444 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-20c49d2d-6adf-4b94-9f2a-e8290fd1f9b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8bac21ce60f4be39defd7fc6611a395', 'request_id': 'req-20c49d2d-6adf-4b94-9f2a-e8290fd1f9b8'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:56.387855 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-20c49d2d-6adf-4b94-9f2a-e8290fd1f9b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8bac21ce60f4be39defd7fc6611a395', 'request_id': 'req-20c49d2d-6adf-4b94-9f2a-e8290fd1f9b8'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:56.400632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-20c49d2d-6adf-4b94-9f2a-e8290fd1f9b8 admin admin] The token cache is being invalidate because project a8bac21ce60f4be39defd7fc6611a395 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:14:56.400923 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-20c49d2d-6adf-4b94-9f2a-e8290fd1f9b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:56.401544 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-20c49d2d-6adf-4b94-9f2a-e8290fd1f9b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:14:56.403142 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1144/2281] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:14:56 2022] DELETE /identity/v3/projects/a8bac21ce60f4be39defd7fc6611a395 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:56.408046 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94fec5-e0c6-48e9-956a-d5dad9e12773 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:56.408405 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94fec5-e0c6-48e9-956a-d5dad9e12773 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:56.408748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94fec5-e0c6-48e9-956a-d5dad9e12773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:56.427022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-3b94fec5-e0c6-48e9-956a-d5dad9e12773 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:56.429066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3b94fec5-e0c6-48e9-956a-d5dad9e12773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:56.447172 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:14:56.494716 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1138/2282] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:56.500397 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1aec9-b225-4648-af34-fcdb741ee047 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:56.500777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1aec9-b225-4648-af34-fcdb741ee047 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:56.501127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1aec9-b225-4648-af34-fcdb741ee047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:56.519587 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-b6a1aec9-b225-4648-af34-fcdb741ee047 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:56.520798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b6a1aec9-b225-4648-af34-fcdb741ee047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:56.529444 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1145/2283] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:56.533774 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94fec5-e0c6-48e9-956a-d5dad9e12773 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:56.546535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94fec5-e0c6-48e9-956a-d5dad9e12773 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:56.558982 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc1871-30a8-4155-9de8-e257305833d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:56.559735 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc1871-30a8-4155-9de8-e257305833d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:56.560157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc1871-30a8-4155-9de8-e257305833d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:56.560493 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc1871-30a8-4155-9de8-e257305833d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:56.561234 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc1871-30a8-4155-9de8-e257305833d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:14:56.561542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc1871-30a8-4155-9de8-e257305833d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:56.563290 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc1871-30a8-4155-9de8-e257305833d1 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:56.578926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1139/2284] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:14:56.583944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1aec9-b225-4648-af34-fcdb741ee047 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:56.596559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1aec9-b225-4648-af34-fcdb741ee047 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:56.601996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6624f-5988-4772-befc-fd53e19022ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:56.603086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6624f-5988-4772-befc-fd53e19022ec admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:56.603507 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6624f-5988-4772-befc-fd53e19022ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:56.603901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6624f-5988-4772-befc-fd53e19022ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:56.604778 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6624f-5988-4772-befc-fd53e19022ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:56.606763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6624f-5988-4772-befc-fd53e19022ec admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:56.643722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1146/2285] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:14:56 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:14:56.648435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc1871-30a8-4155-9de8-e257305833d1 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:56.660613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc1871-30a8-4155-9de8-e257305833d1 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:56.731451 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc74e0b-5e28-4022-97e0-607c4372d9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:56.732438 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc74e0b-5e28-4022-97e0-607c4372d9a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:56.732729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc74e0b-5e28-4022-97e0-607c4372d9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:56.733007 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc74e0b-5e28-4022-97e0-607c4372d9a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:56.733733 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc74e0b-5e28-4022-97e0-607c4372d9a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:56.735774 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc74e0b-5e28-4022-97e0-607c4372d9a9 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:56.769749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1140/2286] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:14:56 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:14:56.775386 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6624f-5988-4772-befc-fd53e19022ec admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:56.787748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6624f-5988-4772-befc-fd53e19022ec admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:56.793854 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdca4d1-a28d-4848-a1f4-d46768453e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:56.794608 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdca4d1-a28d-4848-a1f4-d46768453e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:56.795039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdca4d1-a28d-4848-a1f4-d46768453e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:56.795489 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdca4d1-a28d-4848-a1f4-d46768453e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:56.796512 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdca4d1-a28d-4848-a1f4-d46768453e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:56.798175 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdca4d1-a28d-4848-a1f4-d46768453e20 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:56.814720 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1147/2287] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:14:56.817019 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc74e0b-5e28-4022-97e0-607c4372d9a9 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:56.828865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc74e0b-5e28-4022-97e0-607c4372d9a9 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:56.858056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12c847-408f-4804-af51-b469ac872f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:56.858841 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12c847-408f-4804-af51-b469ac872f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:56.859052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12c847-408f-4804-af51-b469ac872f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:56.859332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12c847-408f-4804-af51-b469ac872f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:56.860077 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12c847-408f-4804-af51-b469ac872f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:56.869635 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12c847-408f-4804-af51-b469ac872f11 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:56.904902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1141/2288] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:14:56.908993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdca4d1-a28d-4848-a1f4-d46768453e20 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:56.921640 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdca4d1-a28d-4848-a1f4-d46768453e20 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:56.928505 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d76b9-299c-414f-a095-8f18b6101d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:56.929714 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d76b9-299c-414f-a095-8f18b6101d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:56.930170 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d76b9-299c-414f-a095-8f18b6101d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:56.930598 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d76b9-299c-414f-a095-8f18b6101d89 admin admin] PATH_INFO: `/v3/projects/c018eceac56a4a8b81e191303c36898a/users/864106cbb7754013bace02099b20d339/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:56.949480 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d76b9-299c-414f-a095-8f18b6101d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c018eceac56a4a8b81e191303c36898a, user_id=864106cbb7754013bace02099b20d339, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:56.953093 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d76b9-299c-414f-a095-8f18b6101d89 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:56.974710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1148/2289] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:14:56 2022] PUT /identity/v3/projects/c018eceac56a4a8b81e191303c36898a/users/864106cbb7754013bace02099b20d339/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:14:56.985920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c522f0-32d3-4126-83df-1a445621413c None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:56.986213 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c522f0-32d3-4126-83df-1a445621413c None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:56.986497 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c522f0-32d3-4126-83df-1a445621413c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:57.021621 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-07c522f0-32d3-4126-83df-1a445621413c None None] MFA Rules not processed for user `864106cbb7754013bace02099b20d339`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:57.023409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-07c522f0-32d3-4126-83df-1a445621413c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:57.091659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1142/2290] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:14:57.103408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d76b9-299c-414f-a095-8f18b6101d89 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:14:57.105310 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b54d76b9-299c-414f-a095-8f18b6101d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:57.164842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d76b9-299c-414f-a095-8f18b6101d89 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:14:57.168533 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1bed9-b4d4-4ee7-a742-2b582274e987 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:14:57.169186 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1bed9-b4d4-4ee7-a742-2b582274e987 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:57.169474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1bed9-b4d4-4ee7-a742-2b582274e987 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:57.169757 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1bed9-b4d4-4ee7-a742-2b582274e987 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:57.178208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1bed9-b4d4-4ee7-a742-2b582274e987 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:14:57.179667 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1bed9-b4d4-4ee7-a742-2b582274e987 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:14:57.188409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1149/2291] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:14:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:14:58.544745 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b98131-2717-45c6-abd7-99e59eb44f0d None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:14:58.545263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b98131-2717-45c6-abd7-99e59eb44f0d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:14:58.545398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b98131-2717-45c6-abd7-99e59eb44f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:14:58.564667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-d9b98131-2717-45c6-abd7-99e59eb44f0d None None] MFA Rules not processed for user `864106cbb7754013bace02099b20d339`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:14:58.566778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d9b98131-2717-45c6-abd7-99e59eb44f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:14:58.574794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1143/2292] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:14:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:15:07.184620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:15:07.958933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541d835-2427-4ec2-823d-6bcedc12caaf None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:07.959193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541d835-2427-4ec2-823d-6bcedc12caaf None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:07.959485 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541d835-2427-4ec2-823d-6bcedc12caaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:07.979586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-9541d835-2427-4ec2-823d-6bcedc12caaf None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:15:07.980918 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9541d835-2427-4ec2-823d-6bcedc12caaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:08.091113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1150/2293] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jul 20 09:15:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:15:08.159525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b98131-2717-45c6-abd7-99e59eb44f0d None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:08.160971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d9b98131-2717-45c6-abd7-99e59eb44f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:08.179222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:15:08.253034 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b98131-2717-45c6-abd7-99e59eb44f0d None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:08.263863 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8df28b-e9be-4f61-b157-799e32ed42c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:08.265509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8df28b-e9be-4f61-b157-799e32ed42c7 service swift] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:08.265868 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8df28b-e9be-4f61-b157-799e32ed42c7 service swift] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:08.266213 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8df28b-e9be-4f61-b157-799e32ed42c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:08.276780 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8df28b-e9be-4f61-b157-799e32ed42c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:08.278494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8df28b-e9be-4f61-b157-799e32ed42c7 service swift] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:08.289204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1144/2294] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:15:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:15:18.082120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=25 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:15:18.285693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=27 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:15:31.747935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5645f3d-cdc9-49e4-b4d7-60c9b18dfb58 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:31.748968 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5645f3d-cdc9-49e4-b4d7-60c9b18dfb58 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:31.749322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5645f3d-cdc9-49e4-b4d7-60c9b18dfb58 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:31.751657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1151/2295] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jul 20 09:15:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:15:31.758663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f234b-cab7-4bb6-9371-be2a9293401a None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:31.759567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f234b-cab7-4bb6-9371-be2a9293401a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:31.759854 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f234b-cab7-4bb6-9371-be2a9293401a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:31.778905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-512f234b-cab7-4bb6-9371-be2a9293401a None None] MFA Rules not processed for user `be08c43f0d0c4a8194596e54500b44ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:15:31.780151 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-512f234b-cab7-4bb6-9371-be2a9293401a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:31.869487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1145/2296] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jul 20 09:15:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:15:31.883138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5645f3d-cdc9-49e4-b4d7-60c9b18dfb58 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:31.885081 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c5645f3d-cdc9-49e4-b4d7-60c9b18dfb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:31.970931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5645f3d-cdc9-49e4-b4d7-60c9b18dfb58 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:31.975144 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dcd5fb-25ab-4050-9079-6b3d3d6dfe4c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb93eb1d25c4410591125b0ab7a73149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:31.975794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcd5fb-25ab-4050-9079-6b3d3d6dfe4c service glance] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:31.976075 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcd5fb-25ab-4050-9079-6b3d3d6dfe4c service glance] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:31.976354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dcd5fb-25ab-4050-9079-6b3d3d6dfe4c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:31.986176 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dcd5fb-25ab-4050-9079-6b3d3d6dfe4c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:31.987639 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dcd5fb-25ab-4050-9079-6b3d3d6dfe4c service glance] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:31.998672 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1152/2297] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:15:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:15:32.064435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343287c-2d8e-4376-a8f5-e663190087dd None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:32.064775 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343287c-2d8e-4376-a8f5-e663190087dd None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:32.065085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343287c-2d8e-4376-a8f5-e663190087dd None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:32.066450 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1146/2298] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jul 20 09:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:15:32.071365 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6317c-220d-4c65-b2de-3e3078787ba5 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:32.071664 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6317c-220d-4c65-b2de-3e3078787ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:32.071973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6317c-220d-4c65-b2de-3e3078787ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:32.090347 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-54e6317c-220d-4c65-b2de-3e3078787ba5 None None] MFA Rules not processed for user `5e5ad4e122ae471cad4cf1b0e3111dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:15:32.091585 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-54e6317c-220d-4c65-b2de-3e3078787ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:32.146387 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1153/2299] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jul 20 09:15:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:15:32.157920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343287c-2d8e-4376-a8f5-e663190087dd None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:32.159602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8343287c-2d8e-4376-a8f5-e663190087dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:32.173698 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343287c-2d8e-4376-a8f5-e663190087dd None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:32.178528 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0299c-ec62-417f-8ba5-83d52f185544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:32.179367 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0299c-ec62-417f-8ba5-83d52f185544 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:32.179651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0299c-ec62-417f-8ba5-83d52f185544 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:32.179932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0299c-ec62-417f-8ba5-83d52f185544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:32.187981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0299c-ec62-417f-8ba5-83d52f185544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:32.189642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0299c-ec62-417f-8ba5-83d52f185544 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:32.197914 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1147/2300] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:15:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:15:32.313516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d61735-89d4-4212-a978-6f1c4fe1fee9 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:32.313809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d61735-89d4-4212-a978-6f1c4fe1fee9 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:32.314092 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d61735-89d4-4212-a978-6f1c4fe1fee9 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:32.315591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1154/2301] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jul 20 09:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:15:32.321029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733c255-e07b-4450-be3b-7f18686dd0be None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:32.321324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733c255-e07b-4450-be3b-7f18686dd0be None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:32.321611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733c255-e07b-4450-be3b-7f18686dd0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:32.340299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-d733c255-e07b-4450-be3b-7f18686dd0be None None] MFA Rules not processed for user `be08c43f0d0c4a8194596e54500b44ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:15:32.341525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d733c255-e07b-4450-be3b-7f18686dd0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:32.351691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1148/2302] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jul 20 09:15:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:15:32.361757 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d61735-89d4-4212-a978-6f1c4fe1fee9 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:32.373436 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d61735-89d4-4212-a978-6f1c4fe1fee9 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:32.377604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf5190-f9b4-4fc0-a1ff-7b71adcdb468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:32.378266 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf5190-f9b4-4fc0-a1ff-7b71adcdb468 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:32.378549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf5190-f9b4-4fc0-a1ff-7b71adcdb468 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:32.378877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf5190-f9b4-4fc0-a1ff-7b71adcdb468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:32.389657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf5190-f9b4-4fc0-a1ff-7b71adcdb468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:32.391116 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf5190-f9b4-4fc0-a1ff-7b71adcdb468 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:32.403589 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1155/2303] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:15:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:15:32.541486 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a7c9b-bc56-4f10-b717-3167f8cf904f None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:32.541779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a7c9b-bc56-4f10-b717-3167f8cf904f None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:32.542086 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a7c9b-bc56-4f10-b717-3167f8cf904f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:32.564463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-556a7c9b-bc56-4f10-b717-3167f8cf904f None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:15:32.565666 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-556a7c9b-bc56-4f10-b717-3167f8cf904f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:32.580702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1149/2304] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jul 20 09:15:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:15:32.597934 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf5190-f9b4-4fc0-a1ff-7b71adcdb468 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:32.613080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf5190-f9b4-4fc0-a1ff-7b71adcdb468 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:32.619173 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c84298-1a41-4bc9-b294-42b33401977b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:32.620316 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c84298-1a41-4bc9-b294-42b33401977b service swift] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:32.620747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c84298-1a41-4bc9-b294-42b33401977b service swift] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:32.621158 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c84298-1a41-4bc9-b294-42b33401977b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:32.632635 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c84298-1a41-4bc9-b294-42b33401977b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:32.634835 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c84298-1a41-4bc9-b294-42b33401977b service swift] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:32.643661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1156/2305] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:15:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:15:33.183943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd91d0-0bdf-4011-ade8-427439347b6e None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:33.184401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd91d0-0bdf-4011-ade8-427439347b6e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:33.184539 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd91d0-0bdf-4011-ade8-427439347b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:33.204400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-42fd91d0-0bdf-4011-ade8-427439347b6e None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:15:33.205634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-42fd91d0-0bdf-4011-ade8-427439347b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:33.213164 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1150/2306] 10.222.0.44 () {58 vars in 967 bytes} [Wed Jul 20 09:15:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:15:33.225420 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c84298-1a41-4bc9-b294-42b33401977b service swift] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:33.235877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c84298-1a41-4bc9-b294-42b33401977b service swift] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:33.241734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf7f84-eaea-44f5-8714-8372e78c9f7a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:33.242393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf7f84-eaea-44f5-8714-8372e78c9f7a service swift] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:33.242697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf7f84-eaea-44f5-8714-8372e78c9f7a service swift] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:33.242983 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf7f84-eaea-44f5-8714-8372e78c9f7a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:33.251712 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf7f84-eaea-44f5-8714-8372e78c9f7a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:33.253290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf7f84-eaea-44f5-8714-8372e78c9f7a service swift] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:33.261971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1157/2307] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:15:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:15:33.456280 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e81c6-c03a-41a0-885c-adc3433bc424 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:33.456576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e81c6-c03a-41a0-885c-adc3433bc424 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:33.456860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e81c6-c03a-41a0-885c-adc3433bc424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:33.474875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-aa0e81c6-c03a-41a0-885c-adc3433bc424 None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:15:33.476071 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-aa0e81c6-c03a-41a0-885c-adc3433bc424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:33.483349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1151/2308] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jul 20 09:15:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:15:33.495601 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf7f84-eaea-44f5-8714-8372e78c9f7a service swift] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:33.506530 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf7f84-eaea-44f5-8714-8372e78c9f7a service swift] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:33.511474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e340d5-7d08-4130-9cb1-7368de1ecafe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:33.512106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e340d5-7d08-4130-9cb1-7368de1ecafe service swift] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:33.512417 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e340d5-7d08-4130-9cb1-7368de1ecafe service swift] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:33.512693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e340d5-7d08-4130-9cb1-7368de1ecafe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:33.521283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e340d5-7d08-4130-9cb1-7368de1ecafe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:33.522971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e340d5-7d08-4130-9cb1-7368de1ecafe service swift] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:33.530833 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1158/2309] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:15:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:15:33.852262 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad8a1c-36f8-4ef8-b996-0cd74eb9fd76 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:33.852597 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad8a1c-36f8-4ef8-b996-0cd74eb9fd76 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:33.852848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dad8a1c-36f8-4ef8-b996-0cd74eb9fd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:33.871766 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-1dad8a1c-36f8-4ef8-b996-0cd74eb9fd76 None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:15:33.872968 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-1dad8a1c-36f8-4ef8-b996-0cd74eb9fd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:33.880136 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1152/2310] 10.222.0.44 () {58 vars in 967 bytes} [Wed Jul 20 09:15:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:15:33.891984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e340d5-7d08-4130-9cb1-7368de1ecafe service swift] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:33.903133 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e340d5-7d08-4130-9cb1-7368de1ecafe service swift] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:33.908188 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4ccf-73b6-4225-9b70-2c1d83057c5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:33.908856 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4ccf-73b6-4225-9b70-2c1d83057c5d service swift] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:33.909141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4ccf-73b6-4225-9b70-2c1d83057c5d service swift] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:33.909422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e4ccf-73b6-4225-9b70-2c1d83057c5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:33.918365 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e4ccf-73b6-4225-9b70-2c1d83057c5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:33.919957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e4ccf-73b6-4225-9b70-2c1d83057c5d service swift] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:33.928179 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1159/2311] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:15:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 09:15:35.707546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb5886-2218-400d-8325-fc445a53d635 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:35.708177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb5886-2218-400d-8325-fc445a53d635 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:35.708177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb5886-2218-400d-8325-fc445a53d635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:35.726411 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-02fb5886-2218-400d-8325-fc445a53d635 None None] MFA Rules not processed for user `7fed4d6dad5b4a06ae84a26b2aab9850`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:15:35.727658 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-02fb5886-2218-400d-8325-fc445a53d635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:35.783382 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1153/2312] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:15:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:15:35.986624 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4ccf-73b6-4225-9b70-2c1d83057c5d service swift] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:36.023239 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:15:36.063872 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e4ccf-73b6-4225-9b70-2c1d83057c5d service swift] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:36.069451 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd8b6e-7edb-496a-b27d-346fd093733e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:36.070335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd8b6e-7edb-496a-b27d-346fd093733e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:36.070618 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd8b6e-7edb-496a-b27d-346fd093733e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:36.071033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd8b6e-7edb-496a-b27d-346fd093733e admin admin] PATH_INFO: `/v3/users/864106cbb7754013bace02099b20d339` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:36.072948 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd8b6e-7edb-496a-b27d-346fd093733e admin admin] RBAC: Authorizing `identity:delete_user(user_id=864106cbb7754013bace02099b20d339)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:36.075379 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd8b6e-7edb-496a-b27d-346fd093733e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:36.129332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-91bd8b6e-7edb-496a-b27d-346fd093733e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '864106cbb7754013bace02099b20d339'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:36.144439 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-91bd8b6e-7edb-496a-b27d-346fd093733e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '864106cbb7754013bace02099b20d339'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:36.153070 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-91bd8b6e-7edb-496a-b27d-346fd093733e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '864106cbb7754013bace02099b20d339'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:36.153937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-91bd8b6e-7edb-496a-b27d-346fd093733e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '864106cbb7754013bace02099b20d339'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:36.162786 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-91bd8b6e-7edb-496a-b27d-346fd093733e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '864106cbb7754013bace02099b20d339'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:36.169006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1160/2313] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:15:35 2022] DELETE /identity/v3/users/864106cbb7754013bace02099b20d339 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:15:36.174915 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96309bd-ddd7-432f-ad32-7be1c4ad39fd None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:36.176185 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96309bd-ddd7-432f-ad32-7be1c4ad39fd None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:36.176729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96309bd-ddd7-432f-ad32-7be1c4ad39fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:36.197754 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-f96309bd-ddd7-432f-ad32-7be1c4ad39fd None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:15:36.199336 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f96309bd-ddd7-432f-ad32-7be1c4ad39fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:36.253162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:15:36.270770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1154/2314] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:15:36.280346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd8b6e-7edb-496a-b27d-346fd093733e admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:36.281886 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-91bd8b6e-7edb-496a-b27d-346fd093733e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:36.355470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd8b6e-7edb-496a-b27d-346fd093733e admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:36.359458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f72e68-b286-415a-9915-d15db7097485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:36.360201 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f72e68-b286-415a-9915-d15db7097485 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:36.360550 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f72e68-b286-415a-9915-d15db7097485 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:36.360893 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f72e68-b286-415a-9915-d15db7097485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:36.371332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f72e68-b286-415a-9915-d15db7097485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:36.373011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f72e68-b286-415a-9915-d15db7097485 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:36.385686 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1161/2315] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:15:36.587731 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96309bd-ddd7-432f-ad32-7be1c4ad39fd None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:36.589239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f96309bd-ddd7-432f-ad32-7be1c4ad39fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:36.616109 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96309bd-ddd7-432f-ad32-7be1c4ad39fd None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:36.636845 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4b3aa-3c04-4c6c-a10d-ff3a15665d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:36.638163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4b3aa-3c04-4c6c-a10d-ff3a15665d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:36.638442 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4b3aa-3c04-4c6c-a10d-ff3a15665d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:36.639324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4b3aa-3c04-4c6c-a10d-ff3a15665d6c admin admin] PATH_INFO: `/v3/projects/c018eceac56a4a8b81e191303c36898a` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:36.639944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4b3aa-3c04-4c6c-a10d-ff3a15665d6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c018eceac56a4a8b81e191303c36898a)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:36.641893 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4b3aa-3c04-4c6c-a10d-ff3a15665d6c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:36.709302 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bbf4b3aa-3c04-4c6c-a10d-ff3a15665d6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c018eceac56a4a8b81e191303c36898a', 'request_id': 'req-bbf4b3aa-3c04-4c6c-a10d-ff3a15665d6c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:36.717479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bbf4b3aa-3c04-4c6c-a10d-ff3a15665d6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c018eceac56a4a8b81e191303c36898a', 'request_id': 'req-bbf4b3aa-3c04-4c6c-a10d-ff3a15665d6c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:36.732687 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bbf4b3aa-3c04-4c6c-a10d-ff3a15665d6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c018eceac56a4a8b81e191303c36898a', 'request_id': 'req-bbf4b3aa-3c04-4c6c-a10d-ff3a15665d6c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:36.746727 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bbf4b3aa-3c04-4c6c-a10d-ff3a15665d6c admin admin] The token cache is being invalidate because project c018eceac56a4a8b81e191303c36898a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:15:36.747400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bbf4b3aa-3c04-4c6c-a10d-ff3a15665d6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:36.748132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bbf4b3aa-3c04-4c6c-a10d-ff3a15665d6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:36.753816 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1155/2316] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:15:36 2022] DELETE /identity/v3/projects/c018eceac56a4a8b81e191303c36898a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:15:36.755320 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65f9dd-c6eb-46a0-aea9-7c0c9cdc7417 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:36.755842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65f9dd-c6eb-46a0-aea9-7c0c9cdc7417 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:36.756349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65f9dd-c6eb-46a0-aea9-7c0c9cdc7417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:36.779049 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-ad65f9dd-c6eb-46a0-aea9-7c0c9cdc7417 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:15:36.780584 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ad65f9dd-c6eb-46a0-aea9-7c0c9cdc7417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:36.871504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:15:36.892453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1162/2317] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:15:36.959675 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09528bd-f740-4f73-9aa3-b25eab6dc4df None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:36.959986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09528bd-f740-4f73-9aa3-b25eab6dc4df None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:36.960219 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09528bd-f740-4f73-9aa3-b25eab6dc4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:36.979993 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a09528bd-f740-4f73-9aa3-b25eab6dc4df None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:15:36.981197 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a09528bd-f740-4f73-9aa3-b25eab6dc4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:36.990221 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1156/2318] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:15:36.994454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65f9dd-c6eb-46a0-aea9-7c0c9cdc7417 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:37.004414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65f9dd-c6eb-46a0-aea9-7c0c9cdc7417 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:37.009343 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b892595-04fb-438e-95f4-af0c3c9a1a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:37.010016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b892595-04fb-438e-95f4-af0c3c9a1a6c admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:37.010422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b892595-04fb-438e-95f4-af0c3c9a1a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:37.010595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b892595-04fb-438e-95f4-af0c3c9a1a6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:37.011330 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b892595-04fb-438e-95f4-af0c3c9a1a6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:15:37.012805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b892595-04fb-438e-95f4-af0c3c9a1a6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:37.013223 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b892595-04fb-438e-95f4-af0c3c9a1a6c admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:37.026710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1163/2319] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:15:37.031574 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09528bd-f740-4f73-9aa3-b25eab6dc4df None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:37.042509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09528bd-f740-4f73-9aa3-b25eab6dc4df None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:37.047661 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eaa414-b114-4a70-9143-d1d249e5dc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:37.048331 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eaa414-b114-4a70-9143-d1d249e5dc5d admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:37.048610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eaa414-b114-4a70-9143-d1d249e5dc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:37.048891 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eaa414-b114-4a70-9143-d1d249e5dc5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:37.049599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eaa414-b114-4a70-9143-d1d249e5dc5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:37.051998 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eaa414-b114-4a70-9143-d1d249e5dc5d admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:37.080613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1157/2320] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:15:37 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:15:37.085321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b892595-04fb-438e-95f4-af0c3c9a1a6c admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:37.100703 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b892595-04fb-438e-95f4-af0c3c9a1a6c admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:37.159964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e14c1-f08e-4eb9-adbd-b2f484400bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:37.161292 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e14c1-f08e-4eb9-adbd-b2f484400bc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:37.161614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e14c1-f08e-4eb9-adbd-b2f484400bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:37.161894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e14c1-f08e-4eb9-adbd-b2f484400bc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:37.162598 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e14c1-f08e-4eb9-adbd-b2f484400bc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:37.164594 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4e14c1-f08e-4eb9-adbd-b2f484400bc3 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:37.185007 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1164/2321] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:15:37 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:15:37.189575 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eaa414-b114-4a70-9143-d1d249e5dc5d admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:37.200769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eaa414-b114-4a70-9143-d1d249e5dc5d admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:37.205397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48371c01-f6b4-45f4-a373-89aefa313531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:37.206061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48371c01-f6b4-45f4-a373-89aefa313531 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:37.206348 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48371c01-f6b4-45f4-a373-89aefa313531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:37.206631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48371c01-f6b4-45f4-a373-89aefa313531 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:37.207891 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48371c01-f6b4-45f4-a373-89aefa313531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:37.209515 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48371c01-f6b4-45f4-a373-89aefa313531 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:37.219817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1158/2322] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:15:37.223728 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e14c1-f08e-4eb9-adbd-b2f484400bc3 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:37.234277 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e14c1-f08e-4eb9-adbd-b2f484400bc3 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:37.238869 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90606e9d-66b5-46b9-91de-9e6235bdcc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:37.239515 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90606e9d-66b5-46b9-91de-9e6235bdcc0a admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:37.239799 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90606e9d-66b5-46b9-91de-9e6235bdcc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:37.240076 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90606e9d-66b5-46b9-91de-9e6235bdcc0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:37.240794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90606e9d-66b5-46b9-91de-9e6235bdcc0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:37.242366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90606e9d-66b5-46b9-91de-9e6235bdcc0a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:37.252220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1165/2323] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:15:37.256691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48371c01-f6b4-45f4-a373-89aefa313531 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:37.266545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48371c01-f6b4-45f4-a373-89aefa313531 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:37.271502 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443dbf0-b852-47f7-bb95-1eebf055fd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:37.272398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443dbf0-b852-47f7-bb95-1eebf055fd26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:37.272677 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443dbf0-b852-47f7-bb95-1eebf055fd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:37.272979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443dbf0-b852-47f7-bb95-1eebf055fd26 admin admin] PATH_INFO: `/v3/projects/9a202c659f5045c7b92d8e32fcb7bbcc/users/3390843c7f8c4bffac9610c59f621be5/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:37.287738 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443dbf0-b852-47f7-bb95-1eebf055fd26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a202c659f5045c7b92d8e32fcb7bbcc, user_id=3390843c7f8c4bffac9610c59f621be5, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:37.290349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443dbf0-b852-47f7-bb95-1eebf055fd26 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:37.303726 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1159/2324] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:15:37 2022] PUT /identity/v3/projects/9a202c659f5045c7b92d8e32fcb7bbcc/users/3390843c7f8c4bffac9610c59f621be5/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:15:37.313519 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5732d2c-c35f-4e0b-93fe-2a14392c2c47 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:37.313810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5732d2c-c35f-4e0b-93fe-2a14392c2c47 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:37.314093 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5732d2c-c35f-4e0b-93fe-2a14392c2c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:37.340514 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-f5732d2c-c35f-4e0b-93fe-2a14392c2c47 None None] MFA Rules not processed for user `3390843c7f8c4bffac9610c59f621be5`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:15:37.341699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f5732d2c-c35f-4e0b-93fe-2a14392c2c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:37.410027 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1166/2325] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:15:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:15:37.421424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443dbf0-b852-47f7-bb95-1eebf055fd26 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:37.423209 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-2443dbf0-b852-47f7-bb95-1eebf055fd26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:37.496606 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443dbf0-b852-47f7-bb95-1eebf055fd26 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:37.501228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1f43e-a930-47c3-8696-50589edfdac6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:37.502122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1f43e-a930-47c3-8696-50589edfdac6 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:37.502429 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1f43e-a930-47c3-8696-50589edfdac6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:37.502708 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1f43e-a930-47c3-8696-50589edfdac6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:37.513186 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1f43e-a930-47c3-8696-50589edfdac6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:37.514864 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1f43e-a930-47c3-8696-50589edfdac6 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:37.534099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1160/2326] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:15:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:15:37.936155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f6551-c417-4dff-b9fa-945ad783f841 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:37.936784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f6551-c417-4dff-b9fa-945ad783f841 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:37.937289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f6551-c417-4dff-b9fa-945ad783f841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:37.972704 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-300f6551-c417-4dff-b9fa-945ad783f841 None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:15:37.975458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-300f6551-c417-4dff-b9fa-945ad783f841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:38.044451 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:15:38.074631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1167/2327] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jul 20 09:15:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:15:38.089621 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1f43e-a930-47c3-8696-50589edfdac6 service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:38.092076 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-73d1f43e-a930-47c3-8696-50589edfdac6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:38.112140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:15:38.196392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1f43e-a930-47c3-8696-50589edfdac6 service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:38.203704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb64859-c0cd-4ede-8b60-8036e1bf5b72 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:38.204370 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb64859-c0cd-4ede-8b60-8036e1bf5b72 service swift] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:38.204651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb64859-c0cd-4ede-8b60-8036e1bf5b72 service swift] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:38.204979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb64859-c0cd-4ede-8b60-8036e1bf5b72 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:38.213934 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb64859-c0cd-4ede-8b60-8036e1bf5b72 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:38.215672 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb64859-c0cd-4ede-8b60-8036e1bf5b72 service swift] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:38.224946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1161/2328] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:15:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:15:48.075313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:15:48.220473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=26 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:15:54.152826 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f6551-c417-4dff-b9fa-945ad783f841 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:54.154311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-300f6551-c417-4dff-b9fa-945ad783f841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:54.226250 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f6551-c417-4dff-b9fa-945ad783f841 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:54.236363 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2a9dd-b82f-44f8-a0e0-e286c598ee13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '066f69a87f5f48c7b20c1e0fcf5c1107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:54.237040 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2a9dd-b82f-44f8-a0e0-e286c598ee13 service placement] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:54.237360 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2a9dd-b82f-44f8-a0e0-e286c598ee13 service placement] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:54.237674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2a9dd-b82f-44f8-a0e0-e286c598ee13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:54.240012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-12e2a9dd-b82f-44f8-a0e0-e286c598ee13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:54.251369 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2a9dd-b82f-44f8-a0e0-e286c598ee13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:54.253032 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2a9dd-b82f-44f8-a0e0-e286c598ee13 service placement] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:54.263131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1168/2329] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:15:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:15:57.088557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb64859-c0cd-4ede-8b60-8036e1bf5b72 service swift] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:57.150477 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb64859-c0cd-4ede-8b60-8036e1bf5b72 service swift] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:57.155148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c44e42-a32e-44fc-ae1f-132e29e8a020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:57.156014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c44e42-a32e-44fc-ae1f-132e29e8a020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:57.156300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c44e42-a32e-44fc-ae1f-132e29e8a020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:57.156581 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c44e42-a32e-44fc-ae1f-132e29e8a020 admin admin] PATH_INFO: `/v3/users/3390843c7f8c4bffac9610c59f621be5` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:57.157765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c44e42-a32e-44fc-ae1f-132e29e8a020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3390843c7f8c4bffac9610c59f621be5)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:57.159756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c44e42-a32e-44fc-ae1f-132e29e8a020 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:57.208592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-59c44e42-a32e-44fc-ae1f-132e29e8a020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3390843c7f8c4bffac9610c59f621be5'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:57.222067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-59c44e42-a32e-44fc-ae1f-132e29e8a020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3390843c7f8c4bffac9610c59f621be5'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:57.229791 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-59c44e42-a32e-44fc-ae1f-132e29e8a020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3390843c7f8c4bffac9610c59f621be5'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:57.230499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-59c44e42-a32e-44fc-ae1f-132e29e8a020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3390843c7f8c4bffac9610c59f621be5'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:57.238303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-59c44e42-a32e-44fc-ae1f-132e29e8a020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3390843c7f8c4bffac9610c59f621be5'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:57.244750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1162/2330] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:15:57 2022] DELETE /identity/v3/users/3390843c7f8c4bffac9610c59f621be5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:15:57.249207 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980234b-527c-482b-89b8-91b9836bbc48 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:57.249682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980234b-527c-482b-89b8-91b9836bbc48 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:57.249851 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980234b-527c-482b-89b8-91b9836bbc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:57.267573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-9980234b-527c-482b-89b8-91b9836bbc48 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:15:57.268731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9980234b-527c-482b-89b8-91b9836bbc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:57.327427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1169/2331] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:15:57.337827 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c44e42-a32e-44fc-ae1f-132e29e8a020 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:57.339345 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-59c44e42-a32e-44fc-ae1f-132e29e8a020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:57.396247 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c44e42-a32e-44fc-ae1f-132e29e8a020 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:57.399826 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7423f-6027-4b59-b095-c8f1b2ef3697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:57.400469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7423f-6027-4b59-b095-c8f1b2ef3697 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:57.400751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7423f-6027-4b59-b095-c8f1b2ef3697 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:57.401030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7423f-6027-4b59-b095-c8f1b2ef3697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:57.410594 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7423f-6027-4b59-b095-c8f1b2ef3697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:57.412341 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7423f-6027-4b59-b095-c8f1b2ef3697 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:57.421895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1163/2332] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:15:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:15:57.703721 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980234b-527c-482b-89b8-91b9836bbc48 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:15:57.705421 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9980234b-527c-482b-89b8-91b9836bbc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:15:57.719402 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980234b-527c-482b-89b8-91b9836bbc48 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:15:57.725776 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c22ca9-ac6f-4ac4-854c-a4068c023ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:15:57.728408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c22ca9-ac6f-4ac4-854c-a4068c023ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:15:57.735397 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c22ca9-ac6f-4ac4-854c-a4068c023ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:15:57.735987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c22ca9-ac6f-4ac4-854c-a4068c023ada admin admin] PATH_INFO: `/v3/projects/9a202c659f5045c7b92d8e32fcb7bbcc` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:15:57.738654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c22ca9-ac6f-4ac4-854c-a4068c023ada admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a202c659f5045c7b92d8e32fcb7bbcc)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:15:57.741647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c22ca9-ac6f-4ac4-854c-a4068c023ada admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:15:57.801079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-30c22ca9-ac6f-4ac4-854c-a4068c023ada admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a202c659f5045c7b92d8e32fcb7bbcc', 'request_id': 'req-30c22ca9-ac6f-4ac4-854c-a4068c023ada'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:57.806787 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-30c22ca9-ac6f-4ac4-854c-a4068c023ada admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a202c659f5045c7b92d8e32fcb7bbcc', 'request_id': 'req-30c22ca9-ac6f-4ac4-854c-a4068c023ada'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:57.812926 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-30c22ca9-ac6f-4ac4-854c-a4068c023ada admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a202c659f5045c7b92d8e32fcb7bbcc', 'request_id': 'req-30c22ca9-ac6f-4ac4-854c-a4068c023ada'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:57.828515 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-30c22ca9-ac6f-4ac4-854c-a4068c023ada admin admin] The token cache is being invalidate because project 9a202c659f5045c7b92d8e32fcb7bbcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:15:57.828801 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-30c22ca9-ac6f-4ac4-854c-a4068c023ada admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:57.829429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-30c22ca9-ac6f-4ac4-854c-a4068c023ada admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:15:57.831005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1170/2333] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:15:57 2022] DELETE /identity/v3/projects/9a202c659f5045c7b92d8e32fcb7bbcc => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:16:04.015770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ec892-f789-4fca-b93f-958ace897a12 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:16:04.016239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ec892-f789-4fca-b93f-958ace897a12 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:16:04.016354 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ec892-f789-4fca-b93f-958ace897a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:16:04.036351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-3d7ec892-f789-4fca-b93f-958ace897a12 None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:16:04.037598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3d7ec892-f789-4fca-b93f-958ace897a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:16:04.054949 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:16:04.099719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1164/2334] 10.222.0.44 () {58 vars in 967 bytes} [Wed Jul 20 09:16:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:16:04.113884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c22ca9-ac6f-4ac4-854c-a4068c023ada admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:16:04.115474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-30c22ca9-ac6f-4ac4-854c-a4068c023ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:16:04.131053 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:16:04.188089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c22ca9-ac6f-4ac4-854c-a4068c023ada admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:16:04.193720 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8244fd5-0ac4-4104-93b8-61fb8d1e05d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:16:04.194373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8244fd5-0ac4-4104-93b8-61fb8d1e05d6 service swift] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:16:04.194663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8244fd5-0ac4-4104-93b8-61fb8d1e05d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:16:04.195251 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8244fd5-0ac4-4104-93b8-61fb8d1e05d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:16:04.203577 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8244fd5-0ac4-4104-93b8-61fb8d1e05d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:16:04.205157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8244fd5-0ac4-4104-93b8-61fb8d1e05d6 service swift] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:16:04.213356 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1171/2335] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:16:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:16:14.092755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=15 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:16:14.210089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=26 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:16:34.717644 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09c58f-7180-4882-b19d-41de214668ac None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:16:34.718013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09c58f-7180-4882-b19d-41de214668ac None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:16:34.718312 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09c58f-7180-4882-b19d-41de214668ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:16:34.738019 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-6c09c58f-7180-4882-b19d-41de214668ac None None] MFA Rules not processed for user `9a020687e95a40f88d521757d8b25619`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:16:34.739209 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6c09c58f-7180-4882-b19d-41de214668ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:16:34.797068 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1165/2336] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:16:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:16:44.790471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=25 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:16:55.330638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf8425-4002-4120-ba31-5fe5f700bfa6 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:16:55.331161 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf8425-4002-4120-ba31-5fe5f700bfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:16:55.331275 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf8425-4002-4120-ba31-5fe5f700bfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:16:55.349588 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-62bf8425-4002-4120-ba31-5fe5f700bfa6 None None] MFA Rules not processed for user `abdb339c5f5740d6aff4dfd1066f9384`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:16:55.350806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-62bf8425-4002-4120-ba31-5fe5f700bfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:16:55.406447 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1172/2337] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:16:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:16:56.562338 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09c58f-7180-4882-b19d-41de214668ac None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:16:56.563852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6c09c58f-7180-4882-b19d-41de214668ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:16:56.629277 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09c58f-7180-4882-b19d-41de214668ac None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:16:56.633701 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e0ecd-a69a-4a9e-ba02-8d2c5143cab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:16:56.634566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e0ecd-a69a-4a9e-ba02-8d2c5143cab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:16:56.634876 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e0ecd-a69a-4a9e-ba02-8d2c5143cab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:16:56.635156 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e0ecd-a69a-4a9e-ba02-8d2c5143cab0 admin admin] PATH_INFO: `/v3/users/7fed4d6dad5b4a06ae84a26b2aab9850` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:16:56.636322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e0ecd-a69a-4a9e-ba02-8d2c5143cab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fed4d6dad5b4a06ae84a26b2aab9850)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:16:56.638226 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e0ecd-a69a-4a9e-ba02-8d2c5143cab0 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:16:56.684170 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4b8e0ecd-a69a-4a9e-ba02-8d2c5143cab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fed4d6dad5b4a06ae84a26b2aab9850'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:16:56.696028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4b8e0ecd-a69a-4a9e-ba02-8d2c5143cab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fed4d6dad5b4a06ae84a26b2aab9850'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:16:56.702943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4b8e0ecd-a69a-4a9e-ba02-8d2c5143cab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fed4d6dad5b4a06ae84a26b2aab9850'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:16:56.703618 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4b8e0ecd-a69a-4a9e-ba02-8d2c5143cab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fed4d6dad5b4a06ae84a26b2aab9850'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:16:56.711210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4b8e0ecd-a69a-4a9e-ba02-8d2c5143cab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fed4d6dad5b4a06ae84a26b2aab9850'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:16:56.717654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1166/2338] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:16:56 2022] DELETE /identity/v3/users/7fed4d6dad5b4a06ae84a26b2aab9850 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:16:56.958413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf8425-4002-4120-ba31-5fe5f700bfa6 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:16:56.969436 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf8425-4002-4120-ba31-5fe5f700bfa6 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:16:57.039560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:16:57.040433 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:16:57.040717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:16:57.040998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc admin admin] PATH_INFO: `/v3/projects/05578c66d5b14af29cc0d860ed978fbf` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:16:57.042184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=05578c66d5b14af29cc0d860ed978fbf)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:16:57.044119 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:16:57.090016 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05578c66d5b14af29cc0d860ed978fbf', 'request_id': 'req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:16:57.096050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05578c66d5b14af29cc0d860ed978fbf', 'request_id': 'req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:16:57.102400 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05578c66d5b14af29cc0d860ed978fbf', 'request_id': 'req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:16:57.115120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc admin admin] The token cache is being invalidate because project 05578c66d5b14af29cc0d860ed978fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:16:57.115410 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:16:57.116031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:16:57.117561 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1173/2339] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:16:56 2022] DELETE /identity/v3/projects/05578c66d5b14af29cc0d860ed978fbf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:17:03.064662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4141f-2aa9-4de3-9e25-7dffe120c391 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:17:03.065175 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4141f-2aa9-4de3-9e25-7dffe120c391 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:17:03.065445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4141f-2aa9-4de3-9e25-7dffe120c391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:17:03.083154 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-0bc4141f-2aa9-4de3-9e25-7dffe120c391 None None] MFA Rules not processed for user `f0547075e10c498b8d13e1859197cea0`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:17:03.084352 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0bc4141f-2aa9-4de3-9e25-7dffe120c391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:17:03.140019 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1167/2340] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:17:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:17:07.112065 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:17:13.134697 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:17:40.046836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:17:40.049170 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:17:40.107472 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e497d-7fd3-4820-b2bd-5d77836a52cc admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:17:40.112042 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3782152-eca6-4b85-9300-502d7a9ffd05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:17:40.112698 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3782152-eca6-4b85-9300-502d7a9ffd05 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:17:40.112981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3782152-eca6-4b85-9300-502d7a9ffd05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:17:40.113259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3782152-eca6-4b85-9300-502d7a9ffd05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:17:40.115167 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-a3782152-eca6-4b85-9300-502d7a9ffd05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:17:40.190071 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3782152-eca6-4b85-9300-502d7a9ffd05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:17:40.191732 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3782152-eca6-4b85-9300-502d7a9ffd05 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:17:40.202793 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1174/2341] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:17:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:17:42.207574 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4141f-2aa9-4de3-9e25-7dffe120c391 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:17:42.209006 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0bc4141f-2aa9-4de3-9e25-7dffe120c391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:17:42.221691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4141f-2aa9-4de3-9e25-7dffe120c391 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:17:42.225318 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa6f23-eddd-48e6-b6da-b8227e36f2f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:17:42.225959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa6f23-eddd-48e6-b6da-b8227e36f2f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:17:42.226246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa6f23-eddd-48e6-b6da-b8227e36f2f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:17:42.226529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa6f23-eddd-48e6-b6da-b8227e36f2f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:17:42.228293 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-74fa6f23-eddd-48e6-b6da-b8227e36f2f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:17:42.242895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa6f23-eddd-48e6-b6da-b8227e36f2f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:17:42.244317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa6f23-eddd-48e6-b6da-b8227e36f2f2 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:17:42.255369 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1168/2342] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:17:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:17:50.197789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=54 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:17:52.250540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=6 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:17:52.582018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3782152-eca6-4b85-9300-502d7a9ffd05 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:17:52.590710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3782152-eca6-4b85-9300-502d7a9ffd05 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:17:52.594844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834a71d-3711-49bd-9447-cab369fc1280 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:17:52.595552 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834a71d-3711-49bd-9447-cab369fc1280 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:17:52.595841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834a71d-3711-49bd-9447-cab369fc1280 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:17:52.596126 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834a71d-3711-49bd-9447-cab369fc1280 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:17:52.598164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6834a71d-3711-49bd-9447-cab369fc1280 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:17:52.672373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834a71d-3711-49bd-9447-cab369fc1280 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:17:52.674833 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834a71d-3711-49bd-9447-cab369fc1280 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:17:52.684141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1175/2343] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:17:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:17:53.509220 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e904abf-2552-47a4-8539-4cf60d60eead None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:17:53.509732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e904abf-2552-47a4-8539-4cf60d60eead None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:17:53.509838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e904abf-2552-47a4-8539-4cf60d60eead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:17:53.528232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-7e904abf-2552-47a4-8539-4cf60d60eead None None] MFA Rules not processed for user `9a020687e95a40f88d521757d8b25619`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:17:53.529426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-7e904abf-2552-47a4-8539-4cf60d60eead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:17:53.587901 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1169/2344] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:17:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:17:53.599317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834a71d-3711-49bd-9447-cab369fc1280 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:17:53.600762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6834a71d-3711-49bd-9447-cab369fc1280 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:17:53.616632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834a71d-3711-49bd-9447-cab369fc1280 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:17:53.621925 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0e39a-7d0a-48d0-a4a6-23f030566047 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:17:53.622567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0e39a-7d0a-48d0-a4a6-23f030566047 service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:17:53.622887 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0e39a-7d0a-48d0-a4a6-23f030566047 service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:17:53.623168 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0e39a-7d0a-48d0-a4a6-23f030566047 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:17:53.624889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-afe0e39a-7d0a-48d0-a4a6-23f030566047 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:17:53.635735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0e39a-7d0a-48d0-a4a6-23f030566047 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:17:53.637297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe0e39a-7d0a-48d0-a4a6-23f030566047 service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:17:53.646425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1176/2345] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:17:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:17:59.220201 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e904abf-2552-47a4-8539-4cf60d60eead None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:17:59.233239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e904abf-2552-47a4-8539-4cf60d60eead None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:17:59.239527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cf30c-3406-4b7e-b5af-1b70fef127bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:17:59.240483 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cf30c-3406-4b7e-b5af-1b70fef127bc service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:17:59.240607 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cf30c-3406-4b7e-b5af-1b70fef127bc service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:17:59.240913 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cf30c-3406-4b7e-b5af-1b70fef127bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:17:59.242953 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ff2cf30c-3406-4b7e-b5af-1b70fef127bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:17:59.258136 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2cf30c-3406-4b7e-b5af-1b70fef127bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:17:59.259792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2cf30c-3406-4b7e-b5af-1b70fef127bc service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:17:59.276465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1170/2346] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:17:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:18:03.642292 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=34 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:03.783985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0e39a-7d0a-48d0-a4a6-23f030566047 service nova] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:03.785384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-afe0e39a-7d0a-48d0-a4a6-23f030566047 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:03.841110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0e39a-7d0a-48d0-a4a6-23f030566047 service nova] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:03.845135 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bfd427-23e4-4997-a2f9-f68b83eaa100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:03.845777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfd427-23e4-4997-a2f9-f68b83eaa100 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:03.846065 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfd427-23e4-4997-a2f9-f68b83eaa100 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:03.846343 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfd427-23e4-4997-a2f9-f68b83eaa100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:03.848350 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b5bfd427-23e4-4997-a2f9-f68b83eaa100 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:03.910549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bfd427-23e4-4997-a2f9-f68b83eaa100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:03.912014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bfd427-23e4-4997-a2f9-f68b83eaa100 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:03.921974 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1177/2347] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:18:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:18:08.200223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cf30c-3406-4b7e-b5af-1b70fef127bc service nova] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:08.202295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ff2cf30c-3406-4b7e-b5af-1b70fef127bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:08.216616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cf30c-3406-4b7e-b5af-1b70fef127bc service nova] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:08.221798 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd04ce0-8bd7-4aaf-afea-7c9d8166a67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:08.222832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd04ce0-8bd7-4aaf-afea-7c9d8166a67f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:08.223177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd04ce0-8bd7-4aaf-afea-7c9d8166a67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:08.223524 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd04ce0-8bd7-4aaf-afea-7c9d8166a67f admin admin] PATH_INFO: `/v3/users/abdb339c5f5740d6aff4dfd1066f9384` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:08.224772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd04ce0-8bd7-4aaf-afea-7c9d8166a67f admin admin] RBAC: Authorizing `identity:delete_user(user_id=abdb339c5f5740d6aff4dfd1066f9384)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:08.226779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd04ce0-8bd7-4aaf-afea-7c9d8166a67f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:08.275348 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cbd04ce0-8bd7-4aaf-afea-7c9d8166a67f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abdb339c5f5740d6aff4dfd1066f9384'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:08.289195 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cbd04ce0-8bd7-4aaf-afea-7c9d8166a67f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abdb339c5f5740d6aff4dfd1066f9384'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:08.296007 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cbd04ce0-8bd7-4aaf-afea-7c9d8166a67f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abdb339c5f5740d6aff4dfd1066f9384'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:08.296723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cbd04ce0-8bd7-4aaf-afea-7c9d8166a67f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abdb339c5f5740d6aff4dfd1066f9384'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:08.305885 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-cbd04ce0-8bd7-4aaf-afea-7c9d8166a67f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abdb339c5f5740d6aff4dfd1066f9384'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:08.312305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1171/2348] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:18:08 2022] DELETE /identity/v3/users/abdb339c5f5740d6aff4dfd1066f9384 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:18:08.500825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bfd427-23e4-4997-a2f9-f68b83eaa100 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:08.512407 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bfd427-23e4-4997-a2f9-f68b83eaa100 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:08.593383 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae2379-888d-4e1b-a805-1239d17bc2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:08.594473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae2379-888d-4e1b-a805-1239d17bc2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:08.595095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae2379-888d-4e1b-a805-1239d17bc2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:08.595456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae2379-888d-4e1b-a805-1239d17bc2b8 admin admin] PATH_INFO: `/v3/projects/19ee1fe6a5044166ba49d2a44e353f52` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:08.596769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae2379-888d-4e1b-a805-1239d17bc2b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19ee1fe6a5044166ba49d2a44e353f52)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:08.599181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae2379-888d-4e1b-a805-1239d17bc2b8 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:08.650950 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-58ae2379-888d-4e1b-a805-1239d17bc2b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19ee1fe6a5044166ba49d2a44e353f52', 'request_id': 'req-58ae2379-888d-4e1b-a805-1239d17bc2b8'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:08.652932 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:08.658560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-58ae2379-888d-4e1b-a805-1239d17bc2b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19ee1fe6a5044166ba49d2a44e353f52', 'request_id': 'req-58ae2379-888d-4e1b-a805-1239d17bc2b8'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:08.665761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-58ae2379-888d-4e1b-a805-1239d17bc2b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19ee1fe6a5044166ba49d2a44e353f52', 'request_id': 'req-58ae2379-888d-4e1b-a805-1239d17bc2b8'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:08.678088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-58ae2379-888d-4e1b-a805-1239d17bc2b8 admin admin] The token cache is being invalidate because project 19ee1fe6a5044166ba49d2a44e353f52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:18:08.678381 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-58ae2379-888d-4e1b-a805-1239d17bc2b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:08.679045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-58ae2379-888d-4e1b-a805-1239d17bc2b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:08.680602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1178/2349] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:18:08 2022] DELETE /identity/v3/projects/19ee1fe6a5044166ba49d2a44e353f52 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:18:08.685699 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e99d3-10e9-4342-87f6-c82ef20b2383 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:08.685997 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e99d3-10e9-4342-87f6-c82ef20b2383 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:08.686278 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e99d3-10e9-4342-87f6-c82ef20b2383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:08.705598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-169e99d3-10e9-4342-87f6-c82ef20b2383 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:08.706861 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-169e99d3-10e9-4342-87f6-c82ef20b2383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:08.770402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1172/2350] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:18:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:18:08.777975 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947fbefd-3855-4ea6-ad36-4f1203f7d2a2 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:08.778269 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947fbefd-3855-4ea6-ad36-4f1203f7d2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:08.778556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947fbefd-3855-4ea6-ad36-4f1203f7d2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:08.797143 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-947fbefd-3855-4ea6-ad36-4f1203f7d2a2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:08.798367 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-947fbefd-3855-4ea6-ad36-4f1203f7d2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:08.806368 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1179/2351] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:18:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:18:08.810684 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e99d3-10e9-4342-87f6-c82ef20b2383 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:08.818507 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:08.827884 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e99d3-10e9-4342-87f6-c82ef20b2383 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:08.833197 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f87ef-979a-4384-aa32-b02cc5001d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:08.833964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f87ef-979a-4384-aa32-b02cc5001d23 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:08.834373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f87ef-979a-4384-aa32-b02cc5001d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:08.834756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f87ef-979a-4384-aa32-b02cc5001d23 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:08.835570 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f87ef-979a-4384-aa32-b02cc5001d23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:18:08.835972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f87ef-979a-4384-aa32-b02cc5001d23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:08.837660 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f87ef-979a-4384-aa32-b02cc5001d23 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:08.851602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1173/2352] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:18:08.855846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947fbefd-3855-4ea6-ad36-4f1203f7d2a2 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:08.866269 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947fbefd-3855-4ea6-ad36-4f1203f7d2a2 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:08.871232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4d99b-e34b-4374-8ab1-75d886c5ef31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:08.871899 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4d99b-e34b-4374-8ab1-75d886c5ef31 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:08.872170 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4d99b-e34b-4374-8ab1-75d886c5ef31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:08.872446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4d99b-e34b-4374-8ab1-75d886c5ef31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:08.873157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4d99b-e34b-4374-8ab1-75d886c5ef31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:08.876007 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4d99b-e34b-4374-8ab1-75d886c5ef31 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:08.911300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1180/2353] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:18:08 2022] POST /identity/v3/projects => generated 357 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:18:08.913534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f87ef-979a-4384-aa32-b02cc5001d23 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:08.929567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f87ef-979a-4384-aa32-b02cc5001d23 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:09.020738 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c98c3-4797-416d-911e-7cf9778a5cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:09.021462 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c98c3-4797-416d-911e-7cf9778a5cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:09.021919 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c98c3-4797-416d-911e-7cf9778a5cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:09.022206 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c98c3-4797-416d-911e-7cf9778a5cb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:09.022955 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c98c3-4797-416d-911e-7cf9778a5cb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:09.026330 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c98c3-4797-416d-911e-7cf9778a5cb0 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:09.062273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1174/2354] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:18:08 2022] POST /identity/v3/users => generated 279 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:18:09.067160 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4d99b-e34b-4374-8ab1-75d886c5ef31 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:09.080204 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4d99b-e34b-4374-8ab1-75d886c5ef31 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:09.086415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7f699-c622-453c-8547-2d6e7050d9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:09.087201 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7f699-c622-453c-8547-2d6e7050d9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:09.087486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7f699-c622-453c-8547-2d6e7050d9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:09.087771 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7f699-c622-453c-8547-2d6e7050d9f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:09.088551 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7f699-c622-453c-8547-2d6e7050d9f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:09.090296 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7f699-c622-453c-8547-2d6e7050d9f1 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:09.102244 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1181/2355] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:18:09.106860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c98c3-4797-416d-911e-7cf9778a5cb0 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:09.118377 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c98c3-4797-416d-911e-7cf9778a5cb0 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:09.127258 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2925467e-92aa-44dc-8bbc-55feccde0f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:09.128332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925467e-92aa-44dc-8bbc-55feccde0f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:09.128332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925467e-92aa-44dc-8bbc-55feccde0f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:09.128527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925467e-92aa-44dc-8bbc-55feccde0f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:09.129915 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925467e-92aa-44dc-8bbc-55feccde0f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:09.131561 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2925467e-92aa-44dc-8bbc-55feccde0f9c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:09.149053 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1175/2356] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:18:09.150012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7f699-c622-453c-8547-2d6e7050d9f1 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:09.160954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7f699-c622-453c-8547-2d6e7050d9f1 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:09.168908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769f3af-26ea-47aa-9a1a-d04fbd22bf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:09.169843 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769f3af-26ea-47aa-9a1a-d04fbd22bf92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:09.170129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769f3af-26ea-47aa-9a1a-d04fbd22bf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:09.170410 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769f3af-26ea-47aa-9a1a-d04fbd22bf92 admin admin] PATH_INFO: `/v3/projects/2c1d9410cbdc461ea2e5bd9fa8339308/users/b13f00430acc4b77a0be8f2e63d18151/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:09.188778 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769f3af-26ea-47aa-9a1a-d04fbd22bf92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c1d9410cbdc461ea2e5bd9fa8339308, user_id=b13f00430acc4b77a0be8f2e63d18151, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:09.191093 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769f3af-26ea-47aa-9a1a-d04fbd22bf92 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:09.212398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1182/2357] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:18:09 2022] PUT /identity/v3/projects/2c1d9410cbdc461ea2e5bd9fa8339308/users/b13f00430acc4b77a0be8f2e63d18151/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:18:09.222973 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f49f21-24b5-4eb1-9fc1-f9ca46300327 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:09.223449 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f49f21-24b5-4eb1-9fc1-f9ca46300327 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:09.223531 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f49f21-24b5-4eb1-9fc1-f9ca46300327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:09.259006 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-51f49f21-24b5-4eb1-9fc1-f9ca46300327 None None] MFA Rules not processed for user `b13f00430acc4b77a0be8f2e63d18151`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:09.260546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-51f49f21-24b5-4eb1-9fc1-f9ca46300327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:09.373914 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1176/2358] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:18:09 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:18:09.378889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769f3af-26ea-47aa-9a1a-d04fbd22bf92 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:09.455274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769f3af-26ea-47aa-9a1a-d04fbd22bf92 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:09.459989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dee3a6-7c91-46eb-aa5b-918614801789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:09.460690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dee3a6-7c91-46eb-aa5b-918614801789 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:09.460996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dee3a6-7c91-46eb-aa5b-918614801789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:09.461263 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dee3a6-7c91-46eb-aa5b-918614801789 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:09.462005 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dee3a6-7c91-46eb-aa5b-918614801789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:09.463966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dee3a6-7c91-46eb-aa5b-918614801789 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:09.494009 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1183/2359] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:18:09 2022] POST /identity/v3/projects => generated 353 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:18:09.495221 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f49f21-24b5-4eb1-9fc1-f9ca46300327 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:09.506706 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f49f21-24b5-4eb1-9fc1-f9ca46300327 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:09.569445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca781db-268b-42ab-be0d-1b4fc4fed2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:09.570053 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca781db-268b-42ab-be0d-1b4fc4fed2db admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:09.570343 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca781db-268b-42ab-be0d-1b4fc4fed2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:09.570623 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca781db-268b-42ab-be0d-1b4fc4fed2db admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:09.571346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca781db-268b-42ab-be0d-1b4fc4fed2db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:09.573456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca781db-268b-42ab-be0d-1b4fc4fed2db admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:09.580719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:09.598498 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1177/2360] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:18:09 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:18:09.599715 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dee3a6-7c91-46eb-aa5b-918614801789 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:09.610164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dee3a6-7c91-46eb-aa5b-918614801789 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:09.615111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0aea4-eb87-41c4-939d-783344525cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:09.615799 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0aea4-eb87-41c4-939d-783344525cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:09.616084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0aea4-eb87-41c4-939d-783344525cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:09.616362 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0aea4-eb87-41c4-939d-783344525cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:09.617090 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f0aea4-eb87-41c4-939d-783344525cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:09.618699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f0aea4-eb87-41c4-939d-783344525cf9 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:09.633819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1184/2361] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:18:09.635294 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca781db-268b-42ab-be0d-1b4fc4fed2db admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:09.646450 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca781db-268b-42ab-be0d-1b4fc4fed2db admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:09.651216 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda67ff-80b8-449d-b9d0-c824d5949a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:09.652184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda67ff-80b8-449d-b9d0-c824d5949a05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:09.652470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda67ff-80b8-449d-b9d0-c824d5949a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:09.652748 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda67ff-80b8-449d-b9d0-c824d5949a05 admin admin] PATH_INFO: `/v3/projects/a4a1e5156f6345948e8dbac87316955d/users/ce97188ddecc463ba8af4406f8be0f7f/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:09.668956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda67ff-80b8-449d-b9d0-c824d5949a05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a1e5156f6345948e8dbac87316955d, user_id=ce97188ddecc463ba8af4406f8be0f7f, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:09.671678 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda67ff-80b8-449d-b9d0-c824d5949a05 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:09.688781 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1178/2362] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:18:09 2022] PUT /identity/v3/projects/a4a1e5156f6345948e8dbac87316955d/users/ce97188ddecc463ba8af4406f8be0f7f/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:18:09.693710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0aea4-eb87-41c4-939d-783344525cf9 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:09.705314 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0aea4-eb87-41c4-939d-783344525cf9 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:09.753973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:09.766422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a61bc-308b-46e0-8485-c79d51ebbee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:09.767147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a61bc-308b-46e0-8485-c79d51ebbee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:09.767435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a61bc-308b-46e0-8485-c79d51ebbee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:09.767717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a61bc-308b-46e0-8485-c79d51ebbee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:09.768465 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a61bc-308b-46e0-8485-c79d51ebbee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:09.770074 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a61bc-308b-46e0-8485-c79d51ebbee2 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:09.781143 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1185/2363] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:18:09.785832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda67ff-80b8-449d-b9d0-c824d5949a05 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:09.796553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda67ff-80b8-449d-b9d0-c824d5949a05 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:09.801809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037615a-987a-4209-a17a-3c94538a293c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:09.802762 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037615a-987a-4209-a17a-3c94538a293c admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:09.803058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037615a-987a-4209-a17a-3c94538a293c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:09.803338 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037615a-987a-4209-a17a-3c94538a293c admin admin] PATH_INFO: `/v3/projects/a4a1e5156f6345948e8dbac87316955d/users/ce97188ddecc463ba8af4406f8be0f7f/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:09.805350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037615a-987a-4209-a17a-3c94538a293c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a1e5156f6345948e8dbac87316955d, user_id=ce97188ddecc463ba8af4406f8be0f7f, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:09.808032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4037615a-987a-4209-a17a-3c94538a293c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:09.819298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1179/2364] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:18:09 2022] PUT /identity/v3/projects/a4a1e5156f6345948e8dbac87316955d/users/ce97188ddecc463ba8af4406f8be0f7f/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:18:09.829520 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3f526-9e6b-40ed-b379-e07c62a5df71 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:09.829819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3f526-9e6b-40ed-b379-e07c62a5df71 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:09.830112 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3f526-9e6b-40ed-b379-e07c62a5df71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:09.858496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-21e3f526-9e6b-40ed-b379-e07c62a5df71 None None] MFA Rules not processed for user `ce97188ddecc463ba8af4406f8be0f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:09.860058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-21e3f526-9e6b-40ed-b379-e07c62a5df71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:09.935165 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1186/2365] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:18:09 2022] POST /identity/v3/auth/tokens => generated 3287 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:18:09.946989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037615a-987a-4209-a17a-3c94538a293c admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:09.948744 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4037615a-987a-4209-a17a-3c94538a293c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:10.015026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037615a-987a-4209-a17a-3c94538a293c admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:10.019332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a884d1-939c-4394-8441-8a61a9402d5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:10.019972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a884d1-939c-4394-8441-8a61a9402d5a service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:10.020251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a884d1-939c-4394-8441-8a61a9402d5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:10.020527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a884d1-939c-4394-8441-8a61a9402d5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:10.029639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a884d1-939c-4394-8441-8a61a9402d5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:10.031120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a884d1-939c-4394-8441-8a61a9402d5a service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:10.041017 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1180/2366] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:18:09 2022] GET /identity/v3/auth/tokens => generated 3287 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:18:10.110717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3f526-9e6b-40ed-b379-e07c62a5df71 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:10.120545 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3f526-9e6b-40ed-b379-e07c62a5df71 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:10.124218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49504b-eb9c-48e2-b169-f7de7b752530 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:10.124855 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49504b-eb9c-48e2-b169-f7de7b752530 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:10.125145 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49504b-eb9c-48e2-b169-f7de7b752530 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:10.125423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49504b-eb9c-48e2-b169-f7de7b752530 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:10.182030 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49504b-eb9c-48e2-b169-f7de7b752530 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:10.183522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49504b-eb9c-48e2-b169-f7de7b752530 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:10.192694 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1187/2367] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:18:10 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:18:12.165898 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603af5e1-c973-4090-a75c-4d6ae4c227ac None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:12.166403 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603af5e1-c973-4090-a75c-4d6ae4c227ac None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:12.166520 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603af5e1-c973-4090-a75c-4d6ae4c227ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:12.184771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-603af5e1-c973-4090-a75c-4d6ae4c227ac None None] MFA Rules not processed for user `0a3ed8bea97e49628311cb45d48ca699`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:12.185973 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-603af5e1-c973-4090-a75c-4d6ae4c227ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:12.240352 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1181/2368] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:18:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:18:12.545147 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49504b-eb9c-48e2-b169-f7de7b752530 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:12.546691 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-1c49504b-eb9c-48e2-b169-f7de7b752530 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:12.605726 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49504b-eb9c-48e2-b169-f7de7b752530 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:12.609902 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b94a4-d0ef-44b6-bf4a-b8c8b58dba63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:12.610638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b94a4-d0ef-44b6-bf4a-b8c8b58dba63 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:12.610951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b94a4-d0ef-44b6-bf4a-b8c8b58dba63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:12.611228 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b94a4-d0ef-44b6-bf4a-b8c8b58dba63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:12.612998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-283b94a4-d0ef-44b6-bf4a-b8c8b58dba63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:12.623112 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b94a4-d0ef-44b6-bf4a-b8c8b58dba63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:12.624651 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b94a4-d0ef-44b6-bf4a-b8c8b58dba63 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:12.633008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1188/2369] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:18:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:18:13.016358 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603af5e1-c973-4090-a75c-4d6ae4c227ac None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:13.017939 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-603af5e1-c973-4090-a75c-4d6ae4c227ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:13.083657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:13.089343 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603af5e1-c973-4090-a75c-4d6ae4c227ac None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:13.095103 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa95189-3b61-4cf0-8694-5117b29167bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '066f69a87f5f48c7b20c1e0fcf5c1107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:13.095751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa95189-3b61-4cf0-8694-5117b29167bb service placement] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:13.096037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa95189-3b61-4cf0-8694-5117b29167bb service placement] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:13.096317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa95189-3b61-4cf0-8694-5117b29167bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:13.098080 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-7fa95189-3b61-4cf0-8694-5117b29167bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:13.109211 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa95189-3b61-4cf0-8694-5117b29167bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:13.111679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa95189-3b61-4cf0-8694-5117b29167bb service placement] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:13.121303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1182/2370] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:18:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:18:15.049189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b94a4-d0ef-44b6-bf4a-b8c8b58dba63 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:15.050659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-283b94a4-d0ef-44b6-bf4a-b8c8b58dba63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:15.061120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b94a4-d0ef-44b6-bf4a-b8c8b58dba63 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:15.065657 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de931819-3ccc-4a29-8e6e-611e5e9abe98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:15.066319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de931819-3ccc-4a29-8e6e-611e5e9abe98 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:15.066605 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de931819-3ccc-4a29-8e6e-611e5e9abe98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:15.066921 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de931819-3ccc-4a29-8e6e-611e5e9abe98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:15.068816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-de931819-3ccc-4a29-8e6e-611e5e9abe98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:15.080510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de931819-3ccc-4a29-8e6e-611e5e9abe98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:15.082057 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de931819-3ccc-4a29-8e6e-611e5e9abe98 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:15.086910 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:15.091601 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1189/2371] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:18:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:18:20.159294 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9176c-1592-4cae-ace8-f5f95422e9c5 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:20.159817 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9176c-1592-4cae-ace8-f5f95422e9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:20.159923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9176c-1592-4cae-ace8-f5f95422e9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:20.177287 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-98c9176c-1592-4cae-ace8-f5f95422e9c5 None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:20.178463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-98c9176c-1592-4cae-ace8-f5f95422e9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:20.240364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1183/2372] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jul 20 09:18:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:18:20.292929 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de931819-3ccc-4a29-8e6e-611e5e9abe98 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:20.294948 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-de931819-3ccc-4a29-8e6e-611e5e9abe98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:20.377751 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de931819-3ccc-4a29-8e6e-611e5e9abe98 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:20.383159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4feb15-6fe4-4159-a428-8886608f84e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:20.383828 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4feb15-6fe4-4159-a428-8886608f84e9 service swift] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:20.384106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4feb15-6fe4-4159-a428-8886608f84e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:20.384384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4feb15-6fe4-4159-a428-8886608f84e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:20.394586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4feb15-6fe4-4159-a428-8886608f84e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:20.396347 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4feb15-6fe4-4159-a428-8886608f84e9 service swift] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:20.406888 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1190/2373] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:18:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:18:25.596605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9176c-1592-4cae-ace8-f5f95422e9c5 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:25.684777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9176c-1592-4cae-ace8-f5f95422e9c5 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:25.690389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1909b1d-69bf-4d25-b0fd-45dd0e9f67ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:25.691469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1909b1d-69bf-4d25-b0fd-45dd0e9f67ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:25.691838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1909b1d-69bf-4d25-b0fd-45dd0e9f67ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:25.692189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1909b1d-69bf-4d25-b0fd-45dd0e9f67ef admin admin] PATH_INFO: `/v3/users/b13f00430acc4b77a0be8f2e63d18151` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:25.693553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1909b1d-69bf-4d25-b0fd-45dd0e9f67ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=b13f00430acc4b77a0be8f2e63d18151)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:25.696785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1909b1d-69bf-4d25-b0fd-45dd0e9f67ef admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:25.778300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-e1909b1d-69bf-4d25-b0fd-45dd0e9f67ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b13f00430acc4b77a0be8f2e63d18151'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:25.808759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-e1909b1d-69bf-4d25-b0fd-45dd0e9f67ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b13f00430acc4b77a0be8f2e63d18151'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:25.821629 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-e1909b1d-69bf-4d25-b0fd-45dd0e9f67ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b13f00430acc4b77a0be8f2e63d18151'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:25.822594 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-e1909b1d-69bf-4d25-b0fd-45dd0e9f67ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b13f00430acc4b77a0be8f2e63d18151'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:25.836416 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-e1909b1d-69bf-4d25-b0fd-45dd0e9f67ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b13f00430acc4b77a0be8f2e63d18151'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:25.844384 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1184/2374] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:18:25 2022] DELETE /identity/v3/users/b13f00430acc4b77a0be8f2e63d18151 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:18:25.848825 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad9015-3054-41be-b047-f3fb62d6a868 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:25.849293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad9015-3054-41be-b047-f3fb62d6a868 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:25.849424 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad9015-3054-41be-b047-f3fb62d6a868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:25.870860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-52ad9015-3054-41be-b047-f3fb62d6a868 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:25.872145 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-52ad9015-3054-41be-b047-f3fb62d6a868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:25.946590 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1191/2375] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:18:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:18:25.957273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1909b1d-69bf-4d25-b0fd-45dd0e9f67ef admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:25.958827 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e1909b1d-69bf-4d25-b0fd-45dd0e9f67ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:25.977746 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:26.047935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1909b1d-69bf-4d25-b0fd-45dd0e9f67ef admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:26.052054 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065900d-41dd-418d-ad09-376988fad4ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:26.052725 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065900d-41dd-418d-ad09-376988fad4ae service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:26.053011 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065900d-41dd-418d-ad09-376988fad4ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:26.053299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065900d-41dd-418d-ad09-376988fad4ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:26.071792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065900d-41dd-418d-ad09-376988fad4ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:26.073271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065900d-41dd-418d-ad09-376988fad4ae service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:26.089430 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1185/2376] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:18:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:18:26.683807 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad9015-3054-41be-b047-f3fb62d6a868 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:26.685513 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-52ad9015-3054-41be-b047-f3fb62d6a868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:26.722641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad9015-3054-41be-b047-f3fb62d6a868 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:26.727354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dd4fa-25ed-474e-9115-0a1b471aa2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:26.728245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dd4fa-25ed-474e-9115-0a1b471aa2aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:26.728527 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dd4fa-25ed-474e-9115-0a1b471aa2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:26.728804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dd4fa-25ed-474e-9115-0a1b471aa2aa admin admin] PATH_INFO: `/v3/projects/2c1d9410cbdc461ea2e5bd9fa8339308` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:26.730037 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497dd4fa-25ed-474e-9115-0a1b471aa2aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c1d9410cbdc461ea2e5bd9fa8339308)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:26.749405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497dd4fa-25ed-474e-9115-0a1b471aa2aa admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:26.864090 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-497dd4fa-25ed-474e-9115-0a1b471aa2aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c1d9410cbdc461ea2e5bd9fa8339308', 'request_id': 'req-497dd4fa-25ed-474e-9115-0a1b471aa2aa'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:26.885590 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-497dd4fa-25ed-474e-9115-0a1b471aa2aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c1d9410cbdc461ea2e5bd9fa8339308', 'request_id': 'req-497dd4fa-25ed-474e-9115-0a1b471aa2aa'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:26.898742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-497dd4fa-25ed-474e-9115-0a1b471aa2aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c1d9410cbdc461ea2e5bd9fa8339308', 'request_id': 'req-497dd4fa-25ed-474e-9115-0a1b471aa2aa'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:26.923751 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-497dd4fa-25ed-474e-9115-0a1b471aa2aa admin admin] The token cache is being invalidate because project 2c1d9410cbdc461ea2e5bd9fa8339308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:18:26.925290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-497dd4fa-25ed-474e-9115-0a1b471aa2aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:26.925773 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-497dd4fa-25ed-474e-9115-0a1b471aa2aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:26.928031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1192/2377] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:18:26 2022] DELETE /identity/v3/projects/2c1d9410cbdc461ea2e5bd9fa8339308 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:18:26.932063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065900d-41dd-418d-ad09-376988fad4ae service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:26.945261 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065900d-41dd-418d-ad09-376988fad4ae service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:27.015904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dd4fa-25ed-474e-9115-0a1b471aa2aa admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:27.017795 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-497dd4fa-25ed-474e-9115-0a1b471aa2aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:27.034943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:27.044642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e631f3-fd3c-4e69-b7c2-45001234c19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:27.049523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e631f3-fd3c-4e69-b7c2-45001234c19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:27.049961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e631f3-fd3c-4e69-b7c2-45001234c19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:27.051103 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e631f3-fd3c-4e69-b7c2-45001234c19f admin admin] PATH_INFO: `/v3/users/ce97188ddecc463ba8af4406f8be0f7f` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:27.052506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e631f3-fd3c-4e69-b7c2-45001234c19f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce97188ddecc463ba8af4406f8be0f7f)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:27.055615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e631f3-fd3c-4e69-b7c2-45001234c19f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:27.123536 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f4e631f3-fd3c-4e69-b7c2-45001234c19f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce97188ddecc463ba8af4406f8be0f7f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:27.139833 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f4e631f3-fd3c-4e69-b7c2-45001234c19f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce97188ddecc463ba8af4406f8be0f7f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:27.147221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dd4fa-25ed-474e-9115-0a1b471aa2aa admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:27.149421 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f4e631f3-fd3c-4e69-b7c2-45001234c19f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce97188ddecc463ba8af4406f8be0f7f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:27.150326 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f4e631f3-fd3c-4e69-b7c2-45001234c19f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce97188ddecc463ba8af4406f8be0f7f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:27.160794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694e632-0739-41d4-8081-38c1874dce7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:27.161888 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694e632-0739-41d4-8081-38c1874dce7c service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:27.162170 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694e632-0739-41d4-8081-38c1874dce7c service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:27.162509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694e632-0739-41d4-8081-38c1874dce7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:27.164766 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-3694e632-0739-41d4-8081-38c1874dce7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:27.166267 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f4e631f3-fd3c-4e69-b7c2-45001234c19f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce97188ddecc463ba8af4406f8be0f7f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:27.174982 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1186/2378] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:18:26 2022] DELETE /identity/v3/users/ce97188ddecc463ba8af4406f8be0f7f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:18:27.179978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3694e632-0739-41d4-8081-38c1874dce7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:27.180483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3694e632-0739-41d4-8081-38c1874dce7c service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:27.204777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1193/2379] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:18:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:18:27.568321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e631f3-fd3c-4e69-b7c2-45001234c19f admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:27.570110 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f4e631f3-fd3c-4e69-b7c2-45001234c19f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:27.628930 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:27.669505 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e631f3-fd3c-4e69-b7c2-45001234c19f admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:27.675218 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d073c-be27-46b4-a788-3cc8031abead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:27.676182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d073c-be27-46b4-a788-3cc8031abead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:27.676543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d073c-be27-46b4-a788-3cc8031abead admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:27.676898 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d073c-be27-46b4-a788-3cc8031abead admin admin] PATH_INFO: `/v3/projects/a4a1e5156f6345948e8dbac87316955d` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:27.678179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d073c-be27-46b4-a788-3cc8031abead admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4a1e5156f6345948e8dbac87316955d)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:27.680278 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d073c-be27-46b4-a788-3cc8031abead admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:27.763096 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-691d073c-be27-46b4-a788-3cc8031abead admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4a1e5156f6345948e8dbac87316955d', 'request_id': 'req-691d073c-be27-46b4-a788-3cc8031abead'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:27.772152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-691d073c-be27-46b4-a788-3cc8031abead admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4a1e5156f6345948e8dbac87316955d', 'request_id': 'req-691d073c-be27-46b4-a788-3cc8031abead'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:27.785297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-691d073c-be27-46b4-a788-3cc8031abead admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4a1e5156f6345948e8dbac87316955d', 'request_id': 'req-691d073c-be27-46b4-a788-3cc8031abead'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:27.820279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-691d073c-be27-46b4-a788-3cc8031abead admin admin] The token cache is being invalidate because project a4a1e5156f6345948e8dbac87316955d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:18:27.823457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-691d073c-be27-46b4-a788-3cc8031abead admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:27.827395 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-691d073c-be27-46b4-a788-3cc8031abead admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:27.831061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1187/2380] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:18:27 2022] DELETE /identity/v3/projects/a4a1e5156f6345948e8dbac87316955d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:18:27.835629 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1010f9a-a2eb-4fd3-8123-5a9c9dea086b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:27.835932 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1010f9a-a2eb-4fd3-8123-5a9c9dea086b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:27.836239 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1010f9a-a2eb-4fd3-8123-5a9c9dea086b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:27.855453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-c1010f9a-a2eb-4fd3-8123-5a9c9dea086b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:27.856680 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c1010f9a-a2eb-4fd3-8123-5a9c9dea086b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:27.928724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1194/2381] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:18:27.937648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53a5e6-0b7c-4ace-aae9-c1dec80b74ca None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:27.938162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53a5e6-0b7c-4ace-aae9-c1dec80b74ca None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:27.938883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53a5e6-0b7c-4ace-aae9-c1dec80b74ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:27.967167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-4e53a5e6-0b7c-4ace-aae9-c1dec80b74ca None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:27.968469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4e53a5e6-0b7c-4ace-aae9-c1dec80b74ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:27.995836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1010f9a-a2eb-4fd3-8123-5a9c9dea086b None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:27.998734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1188/2382] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:18:28.013045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1010f9a-a2eb-4fd3-8123-5a9c9dea086b None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:28.033343 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0baf729-fa59-4fdc-a84c-ab2a62bc1304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:28.034046 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baf729-fa59-4fdc-a84c-ab2a62bc1304 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:28.034328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baf729-fa59-4fdc-a84c-ab2a62bc1304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:28.034606 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0baf729-fa59-4fdc-a84c-ab2a62bc1304 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:28.035390 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0baf729-fa59-4fdc-a84c-ab2a62bc1304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:18:28.035696 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0baf729-fa59-4fdc-a84c-ab2a62bc1304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:28.037311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0baf729-fa59-4fdc-a84c-ab2a62bc1304 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:28.069279 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1195/2383] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:18:28.073571 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53a5e6-0b7c-4ace-aae9-c1dec80b74ca None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:28.084514 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53a5e6-0b7c-4ace-aae9-c1dec80b74ca None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:28.096344 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992fa7a-9ac2-4b33-87e0-d8fec97a4f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:28.097043 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992fa7a-9ac2-4b33-87e0-d8fec97a4f69 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:28.097327 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992fa7a-9ac2-4b33-87e0-d8fec97a4f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:28.097608 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992fa7a-9ac2-4b33-87e0-d8fec97a4f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:28.098308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992fa7a-9ac2-4b33-87e0-d8fec97a4f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:28.101766 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992fa7a-9ac2-4b33-87e0-d8fec97a4f69 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:28.162132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1189/2384] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:18:28 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:18:28.165196 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0baf729-fa59-4fdc-a84c-ab2a62bc1304 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:28.201200 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0baf729-fa59-4fdc-a84c-ab2a62bc1304 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:28.319033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355da042-ad03-4904-b139-b5d2c78619c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:28.319750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355da042-ad03-4904-b139-b5d2c78619c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:28.320045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355da042-ad03-4904-b139-b5d2c78619c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:28.320438 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355da042-ad03-4904-b139-b5d2c78619c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:28.321078 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355da042-ad03-4904-b139-b5d2c78619c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:28.324011 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355da042-ad03-4904-b139-b5d2c78619c0 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:28.349395 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1196/2385] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:18:28 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:18:28.352503 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992fa7a-9ac2-4b33-87e0-d8fec97a4f69 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:28.369573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992fa7a-9ac2-4b33-87e0-d8fec97a4f69 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:28.375954 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3be2fe-1298-4040-b80c-8ef67885784e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:28.376649 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3be2fe-1298-4040-b80c-8ef67885784e admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:28.376941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3be2fe-1298-4040-b80c-8ef67885784e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:28.377220 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3be2fe-1298-4040-b80c-8ef67885784e admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:28.378029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3be2fe-1298-4040-b80c-8ef67885784e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:28.381299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3be2fe-1298-4040-b80c-8ef67885784e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:28.397833 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1190/2386] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:18:28.401541 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355da042-ad03-4904-b139-b5d2c78619c0 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:28.416844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355da042-ad03-4904-b139-b5d2c78619c0 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:28.423588 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d8e4a8-259b-4532-80b6-1e0f1d455b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:28.424267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d8e4a8-259b-4532-80b6-1e0f1d455b8a admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:28.424550 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d8e4a8-259b-4532-80b6-1e0f1d455b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:28.424827 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d8e4a8-259b-4532-80b6-1e0f1d455b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:28.425563 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d8e4a8-259b-4532-80b6-1e0f1d455b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:28.427516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d8e4a8-259b-4532-80b6-1e0f1d455b8a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:28.445180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1197/2387] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:18:28.446990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3be2fe-1298-4040-b80c-8ef67885784e admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:28.463396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3be2fe-1298-4040-b80c-8ef67885784e admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:28.472598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711d4d2-f750-4ed7-9c1f-7751d3ea7833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:28.473520 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711d4d2-f750-4ed7-9c1f-7751d3ea7833 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:28.473807 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711d4d2-f750-4ed7-9c1f-7751d3ea7833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:28.474087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711d4d2-f750-4ed7-9c1f-7751d3ea7833 admin admin] PATH_INFO: `/v3/projects/77bb956064a64a49b60ea8625c4a6e85/users/d78e18ec88a54404b16d55ae070520a8/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:28.498346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711d4d2-f750-4ed7-9c1f-7751d3ea7833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77bb956064a64a49b60ea8625c4a6e85, user_id=d78e18ec88a54404b16d55ae070520a8, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:28.502183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0711d4d2-f750-4ed7-9c1f-7751d3ea7833 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:28.524495 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1191/2388] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:18:28 2022] PUT /identity/v3/projects/77bb956064a64a49b60ea8625c4a6e85/users/d78e18ec88a54404b16d55ae070520a8/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:18:28.532424 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fc2e0-d437-46d4-ad6a-f6ed4dd3ee68 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:28.535785 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fc2e0-d437-46d4-ad6a-f6ed4dd3ee68 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:28.536192 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fc2e0-d437-46d4-ad6a-f6ed4dd3ee68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:28.553472 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:28.595807 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-685fc2e0-d437-46d4-ad6a-f6ed4dd3ee68 None None] MFA Rules not processed for user `d78e18ec88a54404b16d55ae070520a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:28.597347 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-685fc2e0-d437-46d4-ad6a-f6ed4dd3ee68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:28.688202 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1198/2389] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:18:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:18:28.689356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711d4d2-f750-4ed7-9c1f-7751d3ea7833 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:28.769847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711d4d2-f750-4ed7-9c1f-7751d3ea7833 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:28.774692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3559813-4e5a-4897-a9d4-9917d0b9f5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:28.775436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3559813-4e5a-4897-a9d4-9917d0b9f5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:28.775735 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3559813-4e5a-4897-a9d4-9917d0b9f5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:28.775999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3559813-4e5a-4897-a9d4-9917d0b9f5a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:28.776705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3559813-4e5a-4897-a9d4-9917d0b9f5a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:28.778637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3559813-4e5a-4897-a9d4-9917d0b9f5a7 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:28.784507 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:28.814210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1192/2390] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:18:28 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:18:28.815556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fc2e0-d437-46d4-ad6a-f6ed4dd3ee68 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:28.828733 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fc2e0-d437-46d4-ad6a-f6ed4dd3ee68 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:28.904279 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4092ef0-0624-4894-a069-7f957aefb137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:28.904997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4092ef0-0624-4894-a069-7f957aefb137 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:28.905256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4092ef0-0624-4894-a069-7f957aefb137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:28.905536 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4092ef0-0624-4894-a069-7f957aefb137 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:28.908171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4092ef0-0624-4894-a069-7f957aefb137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:28.912413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4092ef0-0624-4894-a069-7f957aefb137 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:28.941200 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1199/2391] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:18:28 2022] POST /identity/v3/users => generated 300 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:18:28.945337 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3559813-4e5a-4897-a9d4-9917d0b9f5a7 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:28.959986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3559813-4e5a-4897-a9d4-9917d0b9f5a7 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:28.969315 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efcf1cd-2fec-48b2-9278-28408fb0f890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:28.970146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcf1cd-2fec-48b2-9278-28408fb0f890 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:28.970459 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcf1cd-2fec-48b2-9278-28408fb0f890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:28.970852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcf1cd-2fec-48b2-9278-28408fb0f890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:28.971598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efcf1cd-2fec-48b2-9278-28408fb0f890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:28.973446 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efcf1cd-2fec-48b2-9278-28408fb0f890 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:28.988749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1193/2392] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:18:28.990182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4092ef0-0624-4894-a069-7f957aefb137 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:29.007015 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4092ef0-0624-4894-a069-7f957aefb137 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:29.012687 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c25134-a435-41bb-9e19-ee3da2992cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:29.013592 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c25134-a435-41bb-9e19-ee3da2992cce admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:29.013879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c25134-a435-41bb-9e19-ee3da2992cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:29.014159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c25134-a435-41bb-9e19-ee3da2992cce admin admin] PATH_INFO: `/v3/projects/35c278a2cdf74f62a4a0b5b01e86e355/users/056b7130a6674284ba169411e012e6ad/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:29.036297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c25134-a435-41bb-9e19-ee3da2992cce admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c278a2cdf74f62a4a0b5b01e86e355, user_id=056b7130a6674284ba169411e012e6ad, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:29.038915 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c25134-a435-41bb-9e19-ee3da2992cce admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:29.057359 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1200/2393] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:18:28 2022] PUT /identity/v3/projects/35c278a2cdf74f62a4a0b5b01e86e355/users/056b7130a6674284ba169411e012e6ad/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:18:29.061388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efcf1cd-2fec-48b2-9278-28408fb0f890 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:29.074838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efcf1cd-2fec-48b2-9278-28408fb0f890 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:29.139243 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51426d3-4e1e-42d6-afc7-7063dcad9df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:29.140176 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51426d3-4e1e-42d6-afc7-7063dcad9df3 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:29.140565 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51426d3-4e1e-42d6-afc7-7063dcad9df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:29.140917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51426d3-4e1e-42d6-afc7-7063dcad9df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:29.141659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51426d3-4e1e-42d6-afc7-7063dcad9df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:29.143319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51426d3-4e1e-42d6-afc7-7063dcad9df3 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:29.154727 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1194/2394] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:18:29.158841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c25134-a435-41bb-9e19-ee3da2992cce admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:29.170446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c25134-a435-41bb-9e19-ee3da2992cce admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:29.176617 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00fb91-bb69-4e18-8852-cee1ee06ef95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:29.177608 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00fb91-bb69-4e18-8852-cee1ee06ef95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:29.177895 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00fb91-bb69-4e18-8852-cee1ee06ef95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:29.178171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00fb91-bb69-4e18-8852-cee1ee06ef95 admin admin] PATH_INFO: `/v3/projects/35c278a2cdf74f62a4a0b5b01e86e355/users/056b7130a6674284ba169411e012e6ad/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:29.180188 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00fb91-bb69-4e18-8852-cee1ee06ef95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c278a2cdf74f62a4a0b5b01e86e355, user_id=056b7130a6674284ba169411e012e6ad, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:29.182831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00fb91-bb69-4e18-8852-cee1ee06ef95 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:29.196820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1201/2395] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:18:29 2022] PUT /identity/v3/projects/35c278a2cdf74f62a4a0b5b01e86e355/users/056b7130a6674284ba169411e012e6ad/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:18:29.209328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90de477-425b-463e-aa9b-aebd60ac7d20 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:29.209828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90de477-425b-463e-aa9b-aebd60ac7d20 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:29.209828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90de477-425b-463e-aa9b-aebd60ac7d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:29.242353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-e90de477-425b-463e-aa9b-aebd60ac7d20 None None] MFA Rules not processed for user `056b7130a6674284ba169411e012e6ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:29.244037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e90de477-425b-463e-aa9b-aebd60ac7d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:29.374444 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1195/2396] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:18:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:18:29.389331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00fb91-bb69-4e18-8852-cee1ee06ef95 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:29.392346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2f00fb91-bb69-4e18-8852-cee1ee06ef95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:29.459953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:29.466871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00fb91-bb69-4e18-8852-cee1ee06ef95 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:29.473668 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421fb44-f482-41d3-a0c3-cb08b1955c46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:29.474471 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421fb44-f482-41d3-a0c3-cb08b1955c46 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:29.474852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421fb44-f482-41d3-a0c3-cb08b1955c46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:29.475342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421fb44-f482-41d3-a0c3-cb08b1955c46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:29.497652 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421fb44-f482-41d3-a0c3-cb08b1955c46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:29.499233 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421fb44-f482-41d3-a0c3-cb08b1955c46 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:29.511104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1202/2397] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:18:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:18:30.237523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90de477-425b-463e-aa9b-aebd60ac7d20 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:30.301021 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:30.309024 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90de477-425b-463e-aa9b-aebd60ac7d20 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:30.320273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e3792-62b2-4f75-b217-821097206d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:30.321192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e3792-62b2-4f75-b217-821097206d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:30.321476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e3792-62b2-4f75-b217-821097206d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:30.321779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e3792-62b2-4f75-b217-821097206d82 admin admin] PATH_INFO: `/v3/users/d78e18ec88a54404b16d55ae070520a8` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:30.323004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e3792-62b2-4f75-b217-821097206d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d78e18ec88a54404b16d55ae070520a8)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:30.325232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e3792-62b2-4f75-b217-821097206d82 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:30.375921 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b50e3792-62b2-4f75-b217-821097206d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd78e18ec88a54404b16d55ae070520a8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:30.389403 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b50e3792-62b2-4f75-b217-821097206d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd78e18ec88a54404b16d55ae070520a8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:30.399308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b50e3792-62b2-4f75-b217-821097206d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd78e18ec88a54404b16d55ae070520a8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:30.400036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b50e3792-62b2-4f75-b217-821097206d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd78e18ec88a54404b16d55ae070520a8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:30.407827 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b50e3792-62b2-4f75-b217-821097206d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd78e18ec88a54404b16d55ae070520a8'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:30.414650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1196/2398] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:18:30 2022] DELETE /identity/v3/users/d78e18ec88a54404b16d55ae070520a8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:18:30.419112 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ad600-53b8-4b19-a3e5-1cb8c7b30260 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:30.419595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ad600-53b8-4b19-a3e5-1cb8c7b30260 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:30.419704 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ad600-53b8-4b19-a3e5-1cb8c7b30260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:30.438647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-a21ad600-53b8-4b19-a3e5-1cb8c7b30260 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:30.439872 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-a21ad600-53b8-4b19-a3e5-1cb8c7b30260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:30.500102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1203/2399] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:18:30.510055 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e3792-62b2-4f75-b217-821097206d82 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:30.511546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b50e3792-62b2-4f75-b217-821097206d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:30.575964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e3792-62b2-4f75-b217-821097206d82 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:30.579785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478119a8-ea74-498f-bddc-d80d0fb25dec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:30.580435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478119a8-ea74-498f-bddc-d80d0fb25dec service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:30.580714 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478119a8-ea74-498f-bddc-d80d0fb25dec service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:30.580987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478119a8-ea74-498f-bddc-d80d0fb25dec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:30.590164 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478119a8-ea74-498f-bddc-d80d0fb25dec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:30.591620 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478119a8-ea74-498f-bddc-d80d0fb25dec service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:30.601624 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1197/2400] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:18:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:18:30.868673 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ad600-53b8-4b19-a3e5-1cb8c7b30260 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:30.870075 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-a21ad600-53b8-4b19-a3e5-1cb8c7b30260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:30.881951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ad600-53b8-4b19-a3e5-1cb8c7b30260 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:30.886636 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92571532-f0da-441c-adb8-85269d42f038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:30.887472 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92571532-f0da-441c-adb8-85269d42f038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:30.887752 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92571532-f0da-441c-adb8-85269d42f038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:30.888402 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92571532-f0da-441c-adb8-85269d42f038 admin admin] PATH_INFO: `/v3/projects/77bb956064a64a49b60ea8625c4a6e85` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:30.889284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92571532-f0da-441c-adb8-85269d42f038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77bb956064a64a49b60ea8625c4a6e85)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:30.891994 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92571532-f0da-441c-adb8-85269d42f038 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:30.942709 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-92571532-f0da-441c-adb8-85269d42f038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77bb956064a64a49b60ea8625c4a6e85', 'request_id': 'req-92571532-f0da-441c-adb8-85269d42f038'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:30.948225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-92571532-f0da-441c-adb8-85269d42f038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77bb956064a64a49b60ea8625c4a6e85', 'request_id': 'req-92571532-f0da-441c-adb8-85269d42f038'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:30.955956 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-92571532-f0da-441c-adb8-85269d42f038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77bb956064a64a49b60ea8625c4a6e85', 'request_id': 'req-92571532-f0da-441c-adb8-85269d42f038'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:30.971409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-92571532-f0da-441c-adb8-85269d42f038 admin admin] The token cache is being invalidate because project 77bb956064a64a49b60ea8625c4a6e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:18:30.971703 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-92571532-f0da-441c-adb8-85269d42f038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:30.972329 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-92571532-f0da-441c-adb8-85269d42f038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:30.974128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1204/2401] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:18:30 2022] DELETE /identity/v3/projects/77bb956064a64a49b60ea8625c4a6e85 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:18:30.977841 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478119a8-ea74-498f-bddc-d80d0fb25dec service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:30.987664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478119a8-ea74-498f-bddc-d80d0fb25dec service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:31.047686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594f8fc-a638-4059-b963-d5efa4b33461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:31.048901 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594f8fc-a638-4059-b963-d5efa4b33461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:31.049185 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594f8fc-a638-4059-b963-d5efa4b33461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:31.049466 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594f8fc-a638-4059-b963-d5efa4b33461 admin admin] PATH_INFO: `/v3/users/056b7130a6674284ba169411e012e6ad` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:31.050666 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594f8fc-a638-4059-b963-d5efa4b33461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=056b7130a6674284ba169411e012e6ad)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:31.052640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594f8fc-a638-4059-b963-d5efa4b33461 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:31.075886 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:31.102389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b594f8fc-a638-4059-b963-d5efa4b33461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '056b7130a6674284ba169411e012e6ad'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:31.114303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b594f8fc-a638-4059-b963-d5efa4b33461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '056b7130a6674284ba169411e012e6ad'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:31.121441 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b594f8fc-a638-4059-b963-d5efa4b33461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '056b7130a6674284ba169411e012e6ad'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:31.122152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b594f8fc-a638-4059-b963-d5efa4b33461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '056b7130a6674284ba169411e012e6ad'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:31.130053 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b594f8fc-a638-4059-b963-d5efa4b33461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '056b7130a6674284ba169411e012e6ad'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:31.136361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1198/2402] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:18:30 2022] DELETE /identity/v3/users/056b7130a6674284ba169411e012e6ad => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:18:31.564665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92571532-f0da-441c-adb8-85269d42f038 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:31.565849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-92571532-f0da-441c-adb8-85269d42f038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:31.580834 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92571532-f0da-441c-adb8-85269d42f038 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:31.638460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940187f-8f4c-4ae9-94f5-8b15fb3606cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:31.639395 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940187f-8f4c-4ae9-94f5-8b15fb3606cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:31.639675 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940187f-8f4c-4ae9-94f5-8b15fb3606cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:31.640505 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940187f-8f4c-4ae9-94f5-8b15fb3606cd admin admin] PATH_INFO: `/v3/projects/35c278a2cdf74f62a4a0b5b01e86e355` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:31.641729 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d940187f-8f4c-4ae9-94f5-8b15fb3606cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=35c278a2cdf74f62a4a0b5b01e86e355)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:31.643650 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d940187f-8f4c-4ae9-94f5-8b15fb3606cd admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:31.660061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:31.693190 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d940187f-8f4c-4ae9-94f5-8b15fb3606cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35c278a2cdf74f62a4a0b5b01e86e355', 'request_id': 'req-d940187f-8f4c-4ae9-94f5-8b15fb3606cd'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:31.700017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d940187f-8f4c-4ae9-94f5-8b15fb3606cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35c278a2cdf74f62a4a0b5b01e86e355', 'request_id': 'req-d940187f-8f4c-4ae9-94f5-8b15fb3606cd'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:31.706944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d940187f-8f4c-4ae9-94f5-8b15fb3606cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35c278a2cdf74f62a4a0b5b01e86e355', 'request_id': 'req-d940187f-8f4c-4ae9-94f5-8b15fb3606cd'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:31.720594 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d940187f-8f4c-4ae9-94f5-8b15fb3606cd admin admin] The token cache is being invalidate because project 35c278a2cdf74f62a4a0b5b01e86e355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:18:31.720886 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d940187f-8f4c-4ae9-94f5-8b15fb3606cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:31.721516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d940187f-8f4c-4ae9-94f5-8b15fb3606cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:18:31.723229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1205/2403] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:18:31 2022] DELETE /identity/v3/projects/35c278a2cdf74f62a4a0b5b01e86e355 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:18:31.729282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1c46a-0dd3-49ad-99a5-8e3d8f7782f9 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:31.729799 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1c46a-0dd3-49ad-99a5-8e3d8f7782f9 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:31.729922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1c46a-0dd3-49ad-99a5-8e3d8f7782f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:31.747894 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a7c1c46a-0dd3-49ad-99a5-8e3d8f7782f9 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:31.749119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a7c1c46a-0dd3-49ad-99a5-8e3d8f7782f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:31.812691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1199/2404] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:18:31.820653 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23330c-d280-44e0-9778-2ded97f26e51 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:31.820950 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23330c-d280-44e0-9778-2ded97f26e51 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:31.821235 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23330c-d280-44e0-9778-2ded97f26e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:31.840681 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-6a23330c-d280-44e0-9778-2ded97f26e51 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:31.841877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6a23330c-d280-44e0-9778-2ded97f26e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:31.850601 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1206/2405] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:18:31.854957 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1c46a-0dd3-49ad-99a5-8e3d8f7782f9 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:31.865265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1c46a-0dd3-49ad-99a5-8e3d8f7782f9 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:31.869775 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea29207-5cf4-4605-b8a9-8db91edb2b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:31.870409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea29207-5cf4-4605-b8a9-8db91edb2b92 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:31.870700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea29207-5cf4-4605-b8a9-8db91edb2b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:31.870990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea29207-5cf4-4605-b8a9-8db91edb2b92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:31.871709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea29207-5cf4-4605-b8a9-8db91edb2b92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:18:31.872012 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea29207-5cf4-4605-b8a9-8db91edb2b92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:31.873585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea29207-5cf4-4605-b8a9-8db91edb2b92 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:31.885768 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1200/2406] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:18:31.889912 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23330c-d280-44e0-9778-2ded97f26e51 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:31.900368 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23330c-d280-44e0-9778-2ded97f26e51 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:31.904902 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5862362-3e6d-42a2-a25b-3a4454622f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:31.905576 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5862362-3e6d-42a2-a25b-3a4454622f26 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:31.906336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5862362-3e6d-42a2-a25b-3a4454622f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:31.906622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5862362-3e6d-42a2-a25b-3a4454622f26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:31.907346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5862362-3e6d-42a2-a25b-3a4454622f26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:31.909219 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5862362-3e6d-42a2-a25b-3a4454622f26 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:31.934553 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1207/2407] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:18:31 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:18:31.939089 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea29207-5cf4-4605-b8a9-8db91edb2b92 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:31.948890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea29207-5cf4-4605-b8a9-8db91edb2b92 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:32.006605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e663f-d897-4a8b-8b10-501194886fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:32.007325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e663f-d897-4a8b-8b10-501194886fce admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:32.007608 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e663f-d897-4a8b-8b10-501194886fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:32.007889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e663f-d897-4a8b-8b10-501194886fce admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:32.008580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e663f-d897-4a8b-8b10-501194886fce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:32.010496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e663f-d897-4a8b-8b10-501194886fce admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:32.029372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1201/2408] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:18:31 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:18:32.034137 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5862362-3e6d-42a2-a25b-3a4454622f26 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:32.044319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5862362-3e6d-42a2-a25b-3a4454622f26 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:32.049988 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15f1b4-7b15-4f92-9ec3-25c009fb7e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:32.050647 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15f1b4-7b15-4f92-9ec3-25c009fb7e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:32.050971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15f1b4-7b15-4f92-9ec3-25c009fb7e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:32.051260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15f1b4-7b15-4f92-9ec3-25c009fb7e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:32.051989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15f1b4-7b15-4f92-9ec3-25c009fb7e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:32.053574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15f1b4-7b15-4f92-9ec3-25c009fb7e48 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:32.064531 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1208/2409] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:18:32.068395 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e663f-d897-4a8b-8b10-501194886fce admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:32.078536 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e663f-d897-4a8b-8b10-501194886fce admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:32.083121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08599e4d-0c42-406e-bcf4-632212a936a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:32.083771 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08599e4d-0c42-406e-bcf4-632212a936a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:32.084056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08599e4d-0c42-406e-bcf4-632212a936a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:32.084361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08599e4d-0c42-406e-bcf4-632212a936a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:32.085080 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08599e4d-0c42-406e-bcf4-632212a936a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:32.086656 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08599e4d-0c42-406e-bcf4-632212a936a2 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:32.096934 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1202/2410] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:18:32.101159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15f1b4-7b15-4f92-9ec3-25c009fb7e48 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:32.111537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15f1b4-7b15-4f92-9ec3-25c009fb7e48 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:32.116062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3346d-cabf-48e8-9ab9-e8dad531a1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:32.116929 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3346d-cabf-48e8-9ab9-e8dad531a1e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:32.117212 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3346d-cabf-48e8-9ab9-e8dad531a1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:32.117504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3346d-cabf-48e8-9ab9-e8dad531a1e4 admin admin] PATH_INFO: `/v3/projects/f80bf212e1494aee8157d1ed4ab12862/users/b2cc888ddc404fbbbae524011b7e50cb/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:32.132234 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3346d-cabf-48e8-9ab9-e8dad531a1e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f80bf212e1494aee8157d1ed4ab12862, user_id=b2cc888ddc404fbbbae524011b7e50cb, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:32.134888 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be3346d-cabf-48e8-9ab9-e8dad531a1e4 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:32.147430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1209/2411] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:18:32 2022] PUT /identity/v3/projects/f80bf212e1494aee8157d1ed4ab12862/users/b2cc888ddc404fbbbae524011b7e50cb/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:18:32.158546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2875843-9d6a-431d-a8b9-6e63a66b393f None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:32.158904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2875843-9d6a-431d-a8b9-6e63a66b393f None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:32.159168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2875843-9d6a-431d-a8b9-6e63a66b393f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:32.165897 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:32.188550 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-b2875843-9d6a-431d-a8b9-6e63a66b393f None None] MFA Rules not processed for user `b2cc888ddc404fbbbae524011b7e50cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:32.189703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b2875843-9d6a-431d-a8b9-6e63a66b393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:32.251734 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1203/2412] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:18:32 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:18:32.262978 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3346d-cabf-48e8-9ab9-e8dad531a1e4 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:32.264668 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-3be3346d-cabf-48e8-9ab9-e8dad531a1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:32.321613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3346d-cabf-48e8-9ab9-e8dad531a1e4 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:32.325505 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4b1b1-41c9-4525-94dd-47774b8f4582 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb93eb1d25c4410591125b0ab7a73149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:32.326148 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4b1b1-41c9-4525-94dd-47774b8f4582 service glance] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:32.326437 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4b1b1-41c9-4525-94dd-47774b8f4582 service glance] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:32.326747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4b1b1-41c9-4525-94dd-47774b8f4582 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:32.335065 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4b1b1-41c9-4525-94dd-47774b8f4582 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:32.336488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4b1b1-41c9-4525-94dd-47774b8f4582 service glance] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:32.345780 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1210/2413] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:18:32 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:18:34.652612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2875843-9d6a-431d-a8b9-6e63a66b393f None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:34.654119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b2875843-9d6a-431d-a8b9-6e63a66b393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:34.743599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2875843-9d6a-431d-a8b9-6e63a66b393f None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:34.749318 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2f806-a7aa-4a40-b3a8-92b87d33d079 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:34.750778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2f806-a7aa-4a40-b3a8-92b87d33d079 service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:34.751068 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2f806-a7aa-4a40-b3a8-92b87d33d079 service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:34.751346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2f806-a7aa-4a40-b3a8-92b87d33d079 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:34.753482 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5ff2f806-a7aa-4a40-b3a8-92b87d33d079 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:34.766912 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2f806-a7aa-4a40-b3a8-92b87d33d079 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:34.768543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2f806-a7aa-4a40-b3a8-92b87d33d079 service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:34.779454 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1204/2414] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:18:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:18:36.782480 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7ba11-2632-474d-92c8-2a259d565f81 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:36.782976 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7ba11-2632-474d-92c8-2a259d565f81 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:36.783095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7ba11-2632-474d-92c8-2a259d565f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:36.801489 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-27e7ba11-2632-474d-92c8-2a259d565f81 None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:36.803004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-27e7ba11-2632-474d-92c8-2a259d565f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:36.849983 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:36.868687 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1211/2415] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jul 20 09:18:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:18:36.880473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2f806-a7aa-4a40-b3a8-92b87d33d079 service nova] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:36.881906 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5ff2f806-a7aa-4a40-b3a8-92b87d33d079 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:36.951123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2f806-a7aa-4a40-b3a8-92b87d33d079 service nova] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:36.957306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e03c05-3975-44d4-874f-0e5c0e4f6e8f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:36.957957 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e03c05-3975-44d4-874f-0e5c0e4f6e8f service swift] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:36.958242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e03c05-3975-44d4-874f-0e5c0e4f6e8f service swift] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:36.958520 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e03c05-3975-44d4-874f-0e5c0e4f6e8f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:36.966875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e03c05-3975-44d4-874f-0e5c0e4f6e8f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:36.969696 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e03c05-3975-44d4-874f-0e5c0e4f6e8f service swift] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:36.978224 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1205/2416] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:18:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:18:46.862047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=6 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:46.974769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=96 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:18:56.801629 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852c194-4112-4567-8b8d-03fde88f3abf None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:56.802141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852c194-4112-4567-8b8d-03fde88f3abf None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:56.802421 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852c194-4112-4567-8b8d-03fde88f3abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:56.821272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-0852c194-4112-4567-8b8d-03fde88f3abf None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:18:56.822740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-0852c194-4112-4567-8b8d-03fde88f3abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:18:56.831412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1212/2417] 10.222.0.44 () {58 vars in 967 bytes} [Wed Jul 20 09:18:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:18:56.846044 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e03c05-3975-44d4-874f-0e5c0e4f6e8f service swift] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:18:56.856392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e03c05-3975-44d4-874f-0e5c0e4f6e8f service swift] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:18:56.861884 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75580548-13e2-40d0-a091-cebab8e33289 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:18:56.862558 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75580548-13e2-40d0-a091-cebab8e33289 service swift] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:18:56.862869 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75580548-13e2-40d0-a091-cebab8e33289 service swift] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:18:56.863147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75580548-13e2-40d0-a091-cebab8e33289 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:18:56.871176 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75580548-13e2-40d0-a091-cebab8e33289 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:18:56.872740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75580548-13e2-40d0-a091-cebab8e33289 service swift] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:18:56.881831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1206/2418] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:18:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:19:06.815583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:06.871671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852c194-4112-4567-8b8d-03fde88f3abf None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:06.878615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=6 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:06.934890 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852c194-4112-4567-8b8d-03fde88f3abf None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:06.940183 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115f1dd-0ad8-4405-aacf-e697d7d68314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:06.941060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115f1dd-0ad8-4405-aacf-e697d7d68314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:06.941346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115f1dd-0ad8-4405-aacf-e697d7d68314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:06.941626 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115f1dd-0ad8-4405-aacf-e697d7d68314 admin admin] PATH_INFO: `/v3/users/b2cc888ddc404fbbbae524011b7e50cb` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:06.942867 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115f1dd-0ad8-4405-aacf-e697d7d68314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2cc888ddc404fbbbae524011b7e50cb)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:06.944782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115f1dd-0ad8-4405-aacf-e697d7d68314 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:06.992794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c115f1dd-0ad8-4405-aacf-e697d7d68314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2cc888ddc404fbbbae524011b7e50cb'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:07.006196 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c115f1dd-0ad8-4405-aacf-e697d7d68314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2cc888ddc404fbbbae524011b7e50cb'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:07.013056 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c115f1dd-0ad8-4405-aacf-e697d7d68314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2cc888ddc404fbbbae524011b7e50cb'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:07.013761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c115f1dd-0ad8-4405-aacf-e697d7d68314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2cc888ddc404fbbbae524011b7e50cb'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:07.021483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c115f1dd-0ad8-4405-aacf-e697d7d68314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2cc888ddc404fbbbae524011b7e50cb'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:07.027597 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1213/2419] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:19:06 2022] DELETE /identity/v3/users/b2cc888ddc404fbbbae524011b7e50cb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:07.032582 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469161af-fb80-4337-b939-a29365c49619 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:07.033051 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469161af-fb80-4337-b939-a29365c49619 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:07.033336 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469161af-fb80-4337-b939-a29365c49619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:07.051501 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-469161af-fb80-4337-b939-a29365c49619 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:07.052747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-469161af-fb80-4337-b939-a29365c49619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:07.115109 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1207/2420] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:19:07.125365 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115f1dd-0ad8-4405-aacf-e697d7d68314 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:07.126836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c115f1dd-0ad8-4405-aacf-e697d7d68314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:07.188868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115f1dd-0ad8-4405-aacf-e697d7d68314 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:07.192631 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36e6cc-4ed0-49ee-9c10-1e432d1f2f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:07.193290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36e6cc-4ed0-49ee-9c10-1e432d1f2f3b service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:07.193578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36e6cc-4ed0-49ee-9c10-1e432d1f2f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:07.193860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36e6cc-4ed0-49ee-9c10-1e432d1f2f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:07.205475 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36e6cc-4ed0-49ee-9c10-1e432d1f2f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:07.207123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36e6cc-4ed0-49ee-9c10-1e432d1f2f3b service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:07.221986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1214/2421] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:19:07.564658 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469161af-fb80-4337-b939-a29365c49619 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:07.566183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-469161af-fb80-4337-b939-a29365c49619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:07.584473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469161af-fb80-4337-b939-a29365c49619 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:07.589441 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad084e4-19a1-43c4-b086-359f81bf73a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:07.590602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad084e4-19a1-43c4-b086-359f81bf73a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:07.591070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad084e4-19a1-43c4-b086-359f81bf73a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:07.591525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad084e4-19a1-43c4-b086-359f81bf73a0 admin admin] PATH_INFO: `/v3/projects/f80bf212e1494aee8157d1ed4ab12862` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:07.595578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad084e4-19a1-43c4-b086-359f81bf73a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f80bf212e1494aee8157d1ed4ab12862)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:07.597926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad084e4-19a1-43c4-b086-359f81bf73a0 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:07.667969 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bad084e4-19a1-43c4-b086-359f81bf73a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f80bf212e1494aee8157d1ed4ab12862', 'request_id': 'req-bad084e4-19a1-43c4-b086-359f81bf73a0'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:07.683814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bad084e4-19a1-43c4-b086-359f81bf73a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f80bf212e1494aee8157d1ed4ab12862', 'request_id': 'req-bad084e4-19a1-43c4-b086-359f81bf73a0'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:07.694521 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bad084e4-19a1-43c4-b086-359f81bf73a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f80bf212e1494aee8157d1ed4ab12862', 'request_id': 'req-bad084e4-19a1-43c4-b086-359f81bf73a0'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:07.720358 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bad084e4-19a1-43c4-b086-359f81bf73a0 admin admin] The token cache is being invalidate because project f80bf212e1494aee8157d1ed4ab12862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:19:07.720639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bad084e4-19a1-43c4-b086-359f81bf73a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:07.726527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bad084e4-19a1-43c4-b086-359f81bf73a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:07.728823 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1208/2422] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:19:07 2022] DELETE /identity/v3/projects/f80bf212e1494aee8157d1ed4ab12862 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:07.734430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d974a-e9f5-4860-84e0-e680610c157f None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:07.734904 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d974a-e9f5-4860-84e0-e680610c157f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:07.735254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d974a-e9f5-4860-84e0-e680610c157f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:07.755457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-fe1d974a-e9f5-4860-84e0-e680610c157f None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:07.756662 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-fe1d974a-e9f5-4860-84e0-e680610c157f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:07.774839 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:07.826198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1215/2423] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:07.834813 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a970f16-6ef5-4e54-be85-9aa03f6df109 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:07.835111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a970f16-6ef5-4e54-be85-9aa03f6df109 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:07.835394 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a970f16-6ef5-4e54-be85-9aa03f6df109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:07.854571 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-4a970f16-6ef5-4e54-be85-9aa03f6df109 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:07.855877 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4a970f16-6ef5-4e54-be85-9aa03f6df109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:07.865026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1209/2424] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:07.869287 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d974a-e9f5-4860-84e0-e680610c157f None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:07.879839 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d974a-e9f5-4860-84e0-e680610c157f None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:07.884580 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f83e60-90d7-415a-ad9b-cd85912479d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:07.885246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f83e60-90d7-415a-ad9b-cd85912479d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:07.885551 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f83e60-90d7-415a-ad9b-cd85912479d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:07.885834 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f83e60-90d7-415a-ad9b-cd85912479d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:07.886565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f83e60-90d7-415a-ad9b-cd85912479d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:19:07.887022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f83e60-90d7-415a-ad9b-cd85912479d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:07.888683 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f83e60-90d7-415a-ad9b-cd85912479d4 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:07.901151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1216/2425] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:19:07.905532 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a970f16-6ef5-4e54-be85-9aa03f6df109 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:07.914764 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a970f16-6ef5-4e54-be85-9aa03f6df109 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:07.920602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05380f90-9822-4850-9672-3707afb3886a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:07.921471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05380f90-9822-4850-9672-3707afb3886a admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:07.921757 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05380f90-9822-4850-9672-3707afb3886a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:07.922041 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05380f90-9822-4850-9672-3707afb3886a admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:07.922767 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05380f90-9822-4850-9672-3707afb3886a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:07.924685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05380f90-9822-4850-9672-3707afb3886a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:07.951196 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1210/2426] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:19:07 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:19:07.955881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f83e60-90d7-415a-ad9b-cd85912479d4 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:07.966711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f83e60-90d7-415a-ad9b-cd85912479d4 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:08.039423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865dc9ec-0a28-48e2-b220-0aa73cd4ce87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:08.040131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865dc9ec-0a28-48e2-b220-0aa73cd4ce87 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:08.040422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865dc9ec-0a28-48e2-b220-0aa73cd4ce87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:08.040708 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865dc9ec-0a28-48e2-b220-0aa73cd4ce87 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:08.042437 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865dc9ec-0a28-48e2-b220-0aa73cd4ce87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:08.044866 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865dc9ec-0a28-48e2-b220-0aa73cd4ce87 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:08.079345 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1217/2427] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:19:07 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:08.083569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05380f90-9822-4850-9672-3707afb3886a admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:08.094843 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05380f90-9822-4850-9672-3707afb3886a admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:08.099481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dd9f4-fe2a-46a9-8d6c-9ee987041b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:08.100159 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dd9f4-fe2a-46a9-8d6c-9ee987041b62 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:08.100439 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dd9f4-fe2a-46a9-8d6c-9ee987041b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:08.100717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dd9f4-fe2a-46a9-8d6c-9ee987041b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:08.101473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7dd9f4-fe2a-46a9-8d6c-9ee987041b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:08.103128 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7dd9f4-fe2a-46a9-8d6c-9ee987041b62 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:08.119852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1211/2428] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:19:08.124845 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865dc9ec-0a28-48e2-b220-0aa73cd4ce87 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:08.135484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865dc9ec-0a28-48e2-b220-0aa73cd4ce87 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:08.142481 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554224c9-447a-4c8a-9e5c-842696f0e431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:08.143544 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554224c9-447a-4c8a-9e5c-842696f0e431 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:08.144030 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554224c9-447a-4c8a-9e5c-842696f0e431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:08.144395 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554224c9-447a-4c8a-9e5c-842696f0e431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:08.145206 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554224c9-447a-4c8a-9e5c-842696f0e431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:08.146989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554224c9-447a-4c8a-9e5c-842696f0e431 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:08.157715 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1218/2429] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:19:08.161907 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dd9f4-fe2a-46a9-8d6c-9ee987041b62 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:08.171996 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dd9f4-fe2a-46a9-8d6c-9ee987041b62 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:08.182119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574dc67-e045-461f-bf29-e50ed7edb1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:08.183180 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574dc67-e045-461f-bf29-e50ed7edb1f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:08.183466 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574dc67-e045-461f-bf29-e50ed7edb1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:08.183747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574dc67-e045-461f-bf29-e50ed7edb1f1 admin admin] PATH_INFO: `/v3/projects/7c29997cd0f9486c90b5ead9215b5026/users/80166ff7bcaf4c1496fdbc483cc88abe/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:08.200831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574dc67-e045-461f-bf29-e50ed7edb1f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c29997cd0f9486c90b5ead9215b5026, user_id=80166ff7bcaf4c1496fdbc483cc88abe, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:08.204346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574dc67-e045-461f-bf29-e50ed7edb1f1 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:08.219270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1212/2430] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:08 2022] PUT /identity/v3/projects/7c29997cd0f9486c90b5ead9215b5026/users/80166ff7bcaf4c1496fdbc483cc88abe/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:08.229507 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702aefa4-3a98-417f-be9d-1dc613996548 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:08.229806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702aefa4-3a98-417f-be9d-1dc613996548 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:08.230090 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702aefa4-3a98-417f-be9d-1dc613996548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:08.258188 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-702aefa4-3a98-417f-be9d-1dc613996548 None None] MFA Rules not processed for user `80166ff7bcaf4c1496fdbc483cc88abe`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:08.259457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-702aefa4-3a98-417f-be9d-1dc613996548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:08.325936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1219/2431] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:08.338103 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574dc67-e045-461f-bf29-e50ed7edb1f1 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:08.339815 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e574dc67-e045-461f-bf29-e50ed7edb1f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:08.361709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:08.406469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574dc67-e045-461f-bf29-e50ed7edb1f1 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:08.410320 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ab394-a430-40fb-9370-aae0f1b1013c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:08.411026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ab394-a430-40fb-9370-aae0f1b1013c service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:08.411315 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ab394-a430-40fb-9370-aae0f1b1013c service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:08.411599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ab394-a430-40fb-9370-aae0f1b1013c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:08.421324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ab394-a430-40fb-9370-aae0f1b1013c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:08.422824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ab394-a430-40fb-9370-aae0f1b1013c service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:08.435336 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1213/2432] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:08.460499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702aefa4-3a98-417f-be9d-1dc613996548 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:08.480723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:08.528878 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702aefa4-3a98-417f-be9d-1dc613996548 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:08.533940 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3b2f7-4d30-4da8-a8d7-c0760c343fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:08.534870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3b2f7-4d30-4da8-a8d7-c0760c343fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:08.535301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3b2f7-4d30-4da8-a8d7-c0760c343fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:08.535592 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3b2f7-4d30-4da8-a8d7-c0760c343fad admin admin] PATH_INFO: `/v3/users/80166ff7bcaf4c1496fdbc483cc88abe` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:08.536857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3b2f7-4d30-4da8-a8d7-c0760c343fad admin admin] RBAC: Authorizing `identity:delete_user(user_id=80166ff7bcaf4c1496fdbc483cc88abe)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:08.538922 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3b2f7-4d30-4da8-a8d7-c0760c343fad admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:08.592074 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-1ef3b2f7-4d30-4da8-a8d7-c0760c343fad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80166ff7bcaf4c1496fdbc483cc88abe'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:08.611085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-1ef3b2f7-4d30-4da8-a8d7-c0760c343fad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80166ff7bcaf4c1496fdbc483cc88abe'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:08.618879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-1ef3b2f7-4d30-4da8-a8d7-c0760c343fad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80166ff7bcaf4c1496fdbc483cc88abe'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:08.619606 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-1ef3b2f7-4d30-4da8-a8d7-c0760c343fad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80166ff7bcaf4c1496fdbc483cc88abe'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:08.629710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-1ef3b2f7-4d30-4da8-a8d7-c0760c343fad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80166ff7bcaf4c1496fdbc483cc88abe'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:08.640195 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1220/2433] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:19:08 2022] DELETE /identity/v3/users/80166ff7bcaf4c1496fdbc483cc88abe => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:08.645905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501d6a2-9214-4bcb-b025-e52ed8e32e70 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:08.646142 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501d6a2-9214-4bcb-b025-e52ed8e32e70 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:08.646484 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501d6a2-9214-4bcb-b025-e52ed8e32e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:08.665458 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-3501d6a2-9214-4bcb-b025-e52ed8e32e70 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:08.666375 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3501d6a2-9214-4bcb-b025-e52ed8e32e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:08.735550 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1214/2434] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:08.749958 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3b2f7-4d30-4da8-a8d7-c0760c343fad admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:08.751697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-1ef3b2f7-4d30-4da8-a8d7-c0760c343fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:08.851855 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3b2f7-4d30-4da8-a8d7-c0760c343fad admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:08.855755 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73cc31-c43f-474a-a45e-85e34cd62b90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:08.856409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73cc31-c43f-474a-a45e-85e34cd62b90 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:08.856695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73cc31-c43f-474a-a45e-85e34cd62b90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:08.856979 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73cc31-c43f-474a-a45e-85e34cd62b90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:08.866932 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73cc31-c43f-474a-a45e-85e34cd62b90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:08.868375 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73cc31-c43f-474a-a45e-85e34cd62b90 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:08.879267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1221/2435] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:09.277568 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501d6a2-9214-4bcb-b025-e52ed8e32e70 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:09.279285 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3501d6a2-9214-4bcb-b025-e52ed8e32e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:09.291359 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501d6a2-9214-4bcb-b025-e52ed8e32e70 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:09.296662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fcc8a-47c3-401a-b539-8fe88f2f10db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:09.297526 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fcc8a-47c3-401a-b539-8fe88f2f10db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:09.297806 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fcc8a-47c3-401a-b539-8fe88f2f10db admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:09.298079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fcc8a-47c3-401a-b539-8fe88f2f10db admin admin] PATH_INFO: `/v3/projects/7c29997cd0f9486c90b5ead9215b5026` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:09.299398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fcc8a-47c3-401a-b539-8fe88f2f10db admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c29997cd0f9486c90b5ead9215b5026)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:09.301301 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fcc8a-47c3-401a-b539-8fe88f2f10db admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:09.366010 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-159fcc8a-47c3-401a-b539-8fe88f2f10db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c29997cd0f9486c90b5ead9215b5026', 'request_id': 'req-159fcc8a-47c3-401a-b539-8fe88f2f10db'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:09.371641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-159fcc8a-47c3-401a-b539-8fe88f2f10db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c29997cd0f9486c90b5ead9215b5026', 'request_id': 'req-159fcc8a-47c3-401a-b539-8fe88f2f10db'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:09.379896 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-159fcc8a-47c3-401a-b539-8fe88f2f10db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c29997cd0f9486c90b5ead9215b5026', 'request_id': 'req-159fcc8a-47c3-401a-b539-8fe88f2f10db'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:09.391984 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-159fcc8a-47c3-401a-b539-8fe88f2f10db admin admin] The token cache is being invalidate because project 7c29997cd0f9486c90b5ead9215b5026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:19:09.392274 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-159fcc8a-47c3-401a-b539-8fe88f2f10db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:09.392908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-159fcc8a-47c3-401a-b539-8fe88f2f10db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:09.394540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1215/2436] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:19:09 2022] DELETE /identity/v3/projects/7c29997cd0f9486c90b5ead9215b5026 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:09.399903 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46960b5-296d-4fcb-9fbe-518af325da62 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:09.400198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46960b5-296d-4fcb-9fbe-518af325da62 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:09.400478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46960b5-296d-4fcb-9fbe-518af325da62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:09.418451 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-e46960b5-296d-4fcb-9fbe-518af325da62 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:09.423328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e46960b5-296d-4fcb-9fbe-518af325da62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:09.461800 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:09.490079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1222/2437] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:09.497710 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1e988-f33e-4763-ad36-42df13f29940 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:09.497939 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1e988-f33e-4763-ad36-42df13f29940 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:09.498881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1e988-f33e-4763-ad36-42df13f29940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:09.517085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-6cb1e988-f33e-4763-ad36-42df13f29940 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:09.518273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6cb1e988-f33e-4763-ad36-42df13f29940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:09.526170 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1216/2438] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:19:09.531134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46960b5-296d-4fcb-9fbe-518af325da62 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:09.540723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46960b5-296d-4fcb-9fbe-518af325da62 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:09.546143 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2607aeb-c558-4906-8b25-f5298df88068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:09.546857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2607aeb-c558-4906-8b25-f5298df88068 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:09.547124 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2607aeb-c558-4906-8b25-f5298df88068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:09.547403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2607aeb-c558-4906-8b25-f5298df88068 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:09.548142 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2607aeb-c558-4906-8b25-f5298df88068 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:19:09.548449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2607aeb-c558-4906-8b25-f5298df88068 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:09.550049 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2607aeb-c558-4906-8b25-f5298df88068 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:09.561962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1223/2439] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:19:09.566473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1e988-f33e-4763-ad36-42df13f29940 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:09.575846 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1e988-f33e-4763-ad36-42df13f29940 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:09.580466 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f659f3-9817-49ac-a02d-966887901090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:09.581111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f659f3-9817-49ac-a02d-966887901090 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:09.581390 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f659f3-9817-49ac-a02d-966887901090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:09.581666 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f659f3-9817-49ac-a02d-966887901090 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:09.582353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f659f3-9817-49ac-a02d-966887901090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:09.584273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f659f3-9817-49ac-a02d-966887901090 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:09.609879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1217/2440] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:19:09 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:09.614188 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2607aeb-c558-4906-8b25-f5298df88068 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:09.624174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2607aeb-c558-4906-8b25-f5298df88068 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:09.686632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa34c96-1a93-49a3-bc1f-d3895a61f1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:09.687347 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa34c96-1a93-49a3-bc1f-d3895a61f1d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:09.687637 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa34c96-1a93-49a3-bc1f-d3895a61f1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:09.687926 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa34c96-1a93-49a3-bc1f-d3895a61f1d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:09.688617 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa34c96-1a93-49a3-bc1f-d3895a61f1d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:09.690564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa34c96-1a93-49a3-bc1f-d3895a61f1d4 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:09.712796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1224/2441] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:19:09 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:09.713917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f659f3-9817-49ac-a02d-966887901090 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:09.725137 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f659f3-9817-49ac-a02d-966887901090 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:09.729772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e95a4-db3d-4776-b30b-fe1fbf907213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:09.730438 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e95a4-db3d-4776-b30b-fe1fbf907213 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:09.730758 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e95a4-db3d-4776-b30b-fe1fbf907213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:09.731046 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e95a4-db3d-4776-b30b-fe1fbf907213 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:09.731778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e95a4-db3d-4776-b30b-fe1fbf907213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:09.733946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e95a4-db3d-4776-b30b-fe1fbf907213 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:09.744560 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1218/2442] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:09.749221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa34c96-1a93-49a3-bc1f-d3895a61f1d4 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:09.759036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa34c96-1a93-49a3-bc1f-d3895a61f1d4 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:09.763575 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b93de-b563-4987-98e0-e5ea7e3382a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:09.764793 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b93de-b563-4987-98e0-e5ea7e3382a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:09.765080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b93de-b563-4987-98e0-e5ea7e3382a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:09.765362 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b93de-b563-4987-98e0-e5ea7e3382a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:09.766127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b93de-b563-4987-98e0-e5ea7e3382a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:09.767740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70b93de-b563-4987-98e0-e5ea7e3382a8 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:09.777814 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1225/2443] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:09.782809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e95a4-db3d-4776-b30b-fe1fbf907213 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:09.792763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e95a4-db3d-4776-b30b-fe1fbf907213 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:09.798035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096045a-a951-48b6-8389-2cb6c41105d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:09.798942 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096045a-a951-48b6-8389-2cb6c41105d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:09.799250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096045a-a951-48b6-8389-2cb6c41105d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:09.799518 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096045a-a951-48b6-8389-2cb6c41105d4 admin admin] PATH_INFO: `/v3/projects/d828bf5b18a64f78882739463a4ae7ca/users/f8b0e66a61544ef192c0c1045081050e/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:09.801767 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:09.816751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096045a-a951-48b6-8389-2cb6c41105d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d828bf5b18a64f78882739463a4ae7ca, user_id=f8b0e66a61544ef192c0c1045081050e, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:09.819433 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096045a-a951-48b6-8389-2cb6c41105d4 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:09.832538 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1219/2444] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:09 2022] PUT /identity/v3/projects/d828bf5b18a64f78882739463a4ae7ca/users/f8b0e66a61544ef192c0c1045081050e/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:09.844666 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c087c10-fb90-4952-ba4c-5e8148cd8bce None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:09.844960 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c087c10-fb90-4952-ba4c-5e8148cd8bce None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:09.845236 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c087c10-fb90-4952-ba4c-5e8148cd8bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:09.871835 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-9c087c10-fb90-4952-ba4c-5e8148cd8bce None None] MFA Rules not processed for user `f8b0e66a61544ef192c0c1045081050e`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:09.873060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9c087c10-fb90-4952-ba4c-5e8148cd8bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:09.936470 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1226/2445] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:09.947017 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096045a-a951-48b6-8389-2cb6c41105d4 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:09.948708 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0096045a-a951-48b6-8389-2cb6c41105d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:10.005629 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096045a-a951-48b6-8389-2cb6c41105d4 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:10.009143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad3948-2ec0-4177-9a95-f4551bfa3cd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:10.009783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad3948-2ec0-4177-9a95-f4551bfa3cd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:10.010063 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad3948-2ec0-4177-9a95-f4551bfa3cd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:10.010364 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad3948-2ec0-4177-9a95-f4551bfa3cd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:10.019157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad3948-2ec0-4177-9a95-f4551bfa3cd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:10.020577 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad3948-2ec0-4177-9a95-f4551bfa3cd1 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:10.029187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1220/2446] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:13.169125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c087c10-fb90-4952-ba4c-5e8148cd8bce None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:13.172570 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9c087c10-fb90-4952-ba4c-5e8148cd8bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:13.239671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:13.262622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c087c10-fb90-4952-ba4c-5e8148cd8bce None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:13.266342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80238d00-664d-4059-8b5e-66dba5640b77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:13.267049 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80238d00-664d-4059-8b5e-66dba5640b77 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:13.267337 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80238d00-664d-4059-8b5e-66dba5640b77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:13.267614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80238d00-664d-4059-8b5e-66dba5640b77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:13.269369 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-80238d00-664d-4059-8b5e-66dba5640b77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:13.338594 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80238d00-664d-4059-8b5e-66dba5640b77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:13.340044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80238d00-664d-4059-8b5e-66dba5640b77 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:13.350721 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1227/2447] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:16.679880 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad3948-2ec0-4177-9a95-f4551bfa3cd1 service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:16.681899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-adad3948-2ec0-4177-9a95-f4551bfa3cd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:16.749334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80238d00-664d-4059-8b5e-66dba5640b77 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:16.751829 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-80238d00-664d-4059-8b5e-66dba5640b77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:16.770002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80238d00-664d-4059-8b5e-66dba5640b77 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:16.775010 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271a3ad-b241-4c16-8ed4-1d5cb0ba5dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:16.775898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271a3ad-b241-4c16-8ed4-1d5cb0ba5dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:16.776186 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271a3ad-b241-4c16-8ed4-1d5cb0ba5dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:16.776467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271a3ad-b241-4c16-8ed4-1d5cb0ba5dd0 admin admin] PATH_INFO: `/v3/users/9a020687e95a40f88d521757d8b25619` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:16.777696 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271a3ad-b241-4c16-8ed4-1d5cb0ba5dd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a020687e95a40f88d521757d8b25619)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:16.780804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271a3ad-b241-4c16-8ed4-1d5cb0ba5dd0 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:16.818136 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad3948-2ec0-4177-9a95-f4551bfa3cd1 service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:16.823934 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450419ed-d25e-4288-8316-be590ae7a4a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '066f69a87f5f48c7b20c1e0fcf5c1107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:16.824931 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450419ed-d25e-4288-8316-be590ae7a4a8 service placement] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:16.825296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450419ed-d25e-4288-8316-be590ae7a4a8 service placement] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:16.825584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450419ed-d25e-4288-8316-be590ae7a4a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:16.828695 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-450419ed-d25e-4288-8316-be590ae7a4a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:16.845151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-3271a3ad-b241-4c16-8ed4-1d5cb0ba5dd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a020687e95a40f88d521757d8b25619'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:16.845462 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450419ed-d25e-4288-8316-be590ae7a4a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:16.848429 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450419ed-d25e-4288-8316-be590ae7a4a8 service placement] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:16.861568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-3271a3ad-b241-4c16-8ed4-1d5cb0ba5dd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a020687e95a40f88d521757d8b25619'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:16.862402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1221/2448] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:16.868812 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-3271a3ad-b241-4c16-8ed4-1d5cb0ba5dd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a020687e95a40f88d521757d8b25619'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:16.869844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-3271a3ad-b241-4c16-8ed4-1d5cb0ba5dd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a020687e95a40f88d521757d8b25619'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:16.877836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-3271a3ad-b241-4c16-8ed4-1d5cb0ba5dd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a020687e95a40f88d521757d8b25619'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:16.884543 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1228/2449] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:19:16 2022] DELETE /identity/v3/users/9a020687e95a40f88d521757d8b25619 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:17.064736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450419ed-d25e-4288-8316-be590ae7a4a8 service placement] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:17.077792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450419ed-d25e-4288-8316-be590ae7a4a8 service placement] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:17.143594 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1e492-40ac-4972-85d2-af243b3ede61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:17.144467 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1e492-40ac-4972-85d2-af243b3ede61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:17.144749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1e492-40ac-4972-85d2-af243b3ede61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:17.145028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1e492-40ac-4972-85d2-af243b3ede61 admin admin] PATH_INFO: `/v3/projects/94fff07a1fa347b2b5ff853a2c4f1a1a` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:17.146209 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1e492-40ac-4972-85d2-af243b3ede61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94fff07a1fa347b2b5ff853a2c4f1a1a)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:17.148157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1e492-40ac-4972-85d2-af243b3ede61 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:17.164589 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:17.195889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-58f1e492-40ac-4972-85d2-af243b3ede61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94fff07a1fa347b2b5ff853a2c4f1a1a', 'request_id': 'req-58f1e492-40ac-4972-85d2-af243b3ede61'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:17.202508 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-58f1e492-40ac-4972-85d2-af243b3ede61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94fff07a1fa347b2b5ff853a2c4f1a1a', 'request_id': 'req-58f1e492-40ac-4972-85d2-af243b3ede61'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:17.209399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-58f1e492-40ac-4972-85d2-af243b3ede61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94fff07a1fa347b2b5ff853a2c4f1a1a', 'request_id': 'req-58f1e492-40ac-4972-85d2-af243b3ede61'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:17.220551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-58f1e492-40ac-4972-85d2-af243b3ede61 admin admin] The token cache is being invalidate because project 94fff07a1fa347b2b5ff853a2c4f1a1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:19:17.220837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-58f1e492-40ac-4972-85d2-af243b3ede61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:17.221462 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-58f1e492-40ac-4972-85d2-af243b3ede61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:17.223018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1222/2450] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:19:17 2022] DELETE /identity/v3/projects/94fff07a1fa347b2b5ff853a2c4f1a1a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:17.228125 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6002e7e-0f6e-4406-8112-697af67aa613 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:17.228447 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6002e7e-0f6e-4406-8112-697af67aa613 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:17.228734 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6002e7e-0f6e-4406-8112-697af67aa613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:17.246861 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-b6002e7e-0f6e-4406-8112-697af67aa613 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:17.249075 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b6002e7e-0f6e-4406-8112-697af67aa613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:17.299911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:17.310817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1229/2451] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:17.318800 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff5e50-dd88-4c69-9b39-0280eaee0f10 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:17.319101 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff5e50-dd88-4c69-9b39-0280eaee0f10 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:17.319450 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff5e50-dd88-4c69-9b39-0280eaee0f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:17.337428 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-c7ff5e50-dd88-4c69-9b39-0280eaee0f10 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:17.338638 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c7ff5e50-dd88-4c69-9b39-0280eaee0f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:17.347009 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1223/2452] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:17.351271 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6002e7e-0f6e-4406-8112-697af67aa613 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:17.361624 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6002e7e-0f6e-4406-8112-697af67aa613 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:17.366163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9032b23-1d30-4906-87e0-8b5bb6cec512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:17.366855 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9032b23-1d30-4906-87e0-8b5bb6cec512 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:17.367144 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9032b23-1d30-4906-87e0-8b5bb6cec512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:17.367688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9032b23-1d30-4906-87e0-8b5bb6cec512 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:17.368435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9032b23-1d30-4906-87e0-8b5bb6cec512 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:19:17.368751 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9032b23-1d30-4906-87e0-8b5bb6cec512 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:17.370339 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9032b23-1d30-4906-87e0-8b5bb6cec512 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:17.382268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1230/2453] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:19:17.386377 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff5e50-dd88-4c69-9b39-0280eaee0f10 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:17.399338 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff5e50-dd88-4c69-9b39-0280eaee0f10 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:17.405879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1cca3d-0229-4bbe-aceb-8d60db76deb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:17.406819 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cca3d-0229-4bbe-aceb-8d60db76deb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:17.407129 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cca3d-0229-4bbe-aceb-8d60db76deb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:17.407527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cca3d-0229-4bbe-aceb-8d60db76deb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:17.408230 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1cca3d-0229-4bbe-aceb-8d60db76deb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:17.410138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1cca3d-0229-4bbe-aceb-8d60db76deb0 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:17.436286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1224/2454] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:19:17 2022] POST /identity/v3/projects => generated 409 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:17.440485 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9032b23-1d30-4906-87e0-8b5bb6cec512 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:17.450531 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9032b23-1d30-4906-87e0-8b5bb6cec512 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:17.506893 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7aa1d6-8e6a-4a4c-8f00-0c34b044f473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:17.507555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7aa1d6-8e6a-4a4c-8f00-0c34b044f473 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:17.507838 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7aa1d6-8e6a-4a4c-8f00-0c34b044f473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:17.508116 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7aa1d6-8e6a-4a4c-8f00-0c34b044f473 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:17.508807 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7aa1d6-8e6a-4a4c-8f00-0c34b044f473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:17.510711 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7aa1d6-8e6a-4a4c-8f00-0c34b044f473 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:17.530945 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1231/2455] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:19:17 2022] POST /identity/v3/users => generated 305 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:17.535007 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1cca3d-0229-4bbe-aceb-8d60db76deb0 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:17.545539 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1cca3d-0229-4bbe-aceb-8d60db76deb0 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:17.550081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec2b86-ea03-47e5-b1ca-bdc7bc5ca57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:17.550781 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec2b86-ea03-47e5-b1ca-bdc7bc5ca57a admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:17.551048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec2b86-ea03-47e5-b1ca-bdc7bc5ca57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:17.551438 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec2b86-ea03-47e5-b1ca-bdc7bc5ca57a admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:17.552164 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec2b86-ea03-47e5-b1ca-bdc7bc5ca57a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:17.553918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec2b86-ea03-47e5-b1ca-bdc7bc5ca57a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:17.564398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1225/2456] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:17.568385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7aa1d6-8e6a-4a4c-8f00-0c34b044f473 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:17.579267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7aa1d6-8e6a-4a4c-8f00-0c34b044f473 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:17.583754 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb81d7b-8b30-497f-a93c-0b6ba340a376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:17.584434 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb81d7b-8b30-497f-a93c-0b6ba340a376 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:17.584720 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb81d7b-8b30-497f-a93c-0b6ba340a376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:17.584997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb81d7b-8b30-497f-a93c-0b6ba340a376 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:17.585716 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb81d7b-8b30-497f-a93c-0b6ba340a376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:17.587355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb81d7b-8b30-497f-a93c-0b6ba340a376 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:17.597740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1232/2457] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:17.601740 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec2b86-ea03-47e5-b1ca-bdc7bc5ca57a admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:17.611511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec2b86-ea03-47e5-b1ca-bdc7bc5ca57a admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:17.616015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d142328-b271-446a-b616-500c97067c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:17.616879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d142328-b271-446a-b616-500c97067c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:17.617155 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d142328-b271-446a-b616-500c97067c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:17.617424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d142328-b271-446a-b616-500c97067c8f admin admin] PATH_INFO: `/v3/projects/eb75cea8b9a14bc1915bb0a43e07a30e/users/5edb42d224d643a9912f06fcf7c598c0/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:17.631458 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d142328-b271-446a-b616-500c97067c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb75cea8b9a14bc1915bb0a43e07a30e, user_id=5edb42d224d643a9912f06fcf7c598c0, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:17.634086 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d142328-b271-446a-b616-500c97067c8f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:17.648012 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1226/2458] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:17 2022] PUT /identity/v3/projects/eb75cea8b9a14bc1915bb0a43e07a30e/users/5edb42d224d643a9912f06fcf7c598c0/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:17.660758 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ab82e-c7a4-4147-a193-5e4573071a28 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:17.661054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ab82e-c7a4-4147-a193-5e4573071a28 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:17.661341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ab82e-c7a4-4147-a193-5e4573071a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:17.688670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-c39ab82e-c7a4-4147-a193-5e4573071a28 None None] MFA Rules not processed for user `5edb42d224d643a9912f06fcf7c598c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:17.689894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c39ab82e-c7a4-4147-a193-5e4573071a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:17.758722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1233/2459] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:17 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:17.763549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d142328-b271-446a-b616-500c97067c8f admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:17.836678 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d142328-b271-446a-b616-500c97067c8f admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:17.841889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1377b7-6818-4376-86bc-f1019d23a08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:17.843340 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1377b7-6818-4376-86bc-f1019d23a08c admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:17.843647 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1377b7-6818-4376-86bc-f1019d23a08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:17.843936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1377b7-6818-4376-86bc-f1019d23a08c admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:17.844744 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1377b7-6818-4376-86bc-f1019d23a08c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:17.846532 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1377b7-6818-4376-86bc-f1019d23a08c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:17.850658 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ab82e-c7a4-4147-a193-5e4573071a28 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:17.852200 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c39ab82e-c7a4-4147-a193-5e4573071a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:17.875359 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1227/2460] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:19:17 2022] POST /identity/v3/projects => generated 409 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:17.879819 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1377b7-6818-4376-86bc-f1019d23a08c admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:17.921073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:17.930289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ab82e-c7a4-4147-a193-5e4573071a28 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:17.942287 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:17.963675 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1377b7-6818-4376-86bc-f1019d23a08c admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:17.968579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234a168-5953-4530-96c4-5dad7196338c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:17.969256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234a168-5953-4530-96c4-5dad7196338c admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:17.969543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234a168-5953-4530-96c4-5dad7196338c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:17.969825 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234a168-5953-4530-96c4-5dad7196338c admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:17.970517 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234a168-5953-4530-96c4-5dad7196338c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:17.972487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c234a168-5953-4530-96c4-5dad7196338c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:17.991165 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4ef05-770b-49fb-b5e1-00dc729d4207 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:17.992123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4ef05-770b-49fb-b5e1-00dc729d4207 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:17.992123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4ef05-770b-49fb-b5e1-00dc729d4207 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:17.992394 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4ef05-770b-49fb-b5e1-00dc729d4207 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:17.992814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1228/2461] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:19:17 2022] POST /identity/v3/users => generated 311 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:19:17.994285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-fbd4ef05-770b-49fb-b5e1-00dc729d4207 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:17.996932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234a168-5953-4530-96c4-5dad7196338c admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:18.008413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234a168-5953-4530-96c4-5dad7196338c admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:18.015849 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b7b76-03e8-44aa-8554-844b226b451a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:18.016504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b7b76-03e8-44aa-8554-844b226b451a admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:18.016787 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b7b76-03e8-44aa-8554-844b226b451a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:18.017061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b7b76-03e8-44aa-8554-844b226b451a admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:18.017779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b7b76-03e8-44aa-8554-844b226b451a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:18.019532 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b7b76-03e8-44aa-8554-844b226b451a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:18.031891 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1229/2462] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:18.037181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b7b76-03e8-44aa-8554-844b226b451a admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:18.047796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b7b76-03e8-44aa-8554-844b226b451a admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:18.052958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463def42-79d5-4a8e-928e-676c1b59a0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:18.053842 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463def42-79d5-4a8e-928e-676c1b59a0eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:18.054193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463def42-79d5-4a8e-928e-676c1b59a0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:18.054480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463def42-79d5-4a8e-928e-676c1b59a0eb admin admin] PATH_INFO: `/v3/projects/d32dd1dc692245deaf04a07ed0f3bb93/users/ee31547c783f41839bb0b683781ff326/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:18.063836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4ef05-770b-49fb-b5e1-00dc729d4207 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:18.065348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4ef05-770b-49fb-b5e1-00dc729d4207 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:18.071172 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463def42-79d5-4a8e-928e-676c1b59a0eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d32dd1dc692245deaf04a07ed0f3bb93, user_id=ee31547c783f41839bb0b683781ff326, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:18.074090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463def42-79d5-4a8e-928e-676c1b59a0eb admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:18.077900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1234/2463] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:19:18.093319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1230/2464] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:18 2022] PUT /identity/v3/projects/d32dd1dc692245deaf04a07ed0f3bb93/users/ee31547c783f41839bb0b683781ff326/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:19:18.095425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4ef05-770b-49fb-b5e1-00dc729d4207 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:18.108848 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4ef05-770b-49fb-b5e1-00dc729d4207 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:18.184524 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cde95d-e867-4eb2-9e7f-e066dddc1288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:18.185205 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cde95d-e867-4eb2-9e7f-e066dddc1288 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:18.185622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cde95d-e867-4eb2-9e7f-e066dddc1288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:18.185921 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cde95d-e867-4eb2-9e7f-e066dddc1288 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:18.186737 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cde95d-e867-4eb2-9e7f-e066dddc1288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:18.188385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cde95d-e867-4eb2-9e7f-e066dddc1288 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:18.198320 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1235/2465] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:18.203383 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463def42-79d5-4a8e-928e-676c1b59a0eb admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:18.213898 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463def42-79d5-4a8e-928e-676c1b59a0eb admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:18.219851 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9424cf-7a66-42f6-aed6-b891009edb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:18.220733 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9424cf-7a66-42f6-aed6-b891009edb7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:18.221024 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9424cf-7a66-42f6-aed6-b891009edb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:18.221291 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9424cf-7a66-42f6-aed6-b891009edb7e admin admin] PATH_INFO: `/v3/projects/d32dd1dc692245deaf04a07ed0f3bb93/users/ee31547c783f41839bb0b683781ff326/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:18.223309 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9424cf-7a66-42f6-aed6-b891009edb7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d32dd1dc692245deaf04a07ed0f3bb93, user_id=ee31547c783f41839bb0b683781ff326, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:18.225938 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9424cf-7a66-42f6-aed6-b891009edb7e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:18.236834 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1231/2466] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:18 2022] PUT /identity/v3/projects/d32dd1dc692245deaf04a07ed0f3bb93/users/ee31547c783f41839bb0b683781ff326/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:18.246814 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734a15d-fefd-4089-bd3d-0875f5caab80 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:18.247119 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734a15d-fefd-4089-bd3d-0875f5caab80 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:18.247405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734a15d-fefd-4089-bd3d-0875f5caab80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:18.273759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-5734a15d-fefd-4089-bd3d-0875f5caab80 None None] MFA Rules not processed for user `ee31547c783f41839bb0b683781ff326`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:18.275002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5734a15d-fefd-4089-bd3d-0875f5caab80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:18.320389 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:18.343218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1236/2467] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:18 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:18.353979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9424cf-7a66-42f6-aed6-b891009edb7e admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:18.355753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4f9424cf-7a66-42f6-aed6-b891009edb7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:18.416806 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9424cf-7a66-42f6-aed6-b891009edb7e admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:18.420557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cdd2cd-fc1e-499c-90d9-6560fe139ab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:18.421193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdd2cd-fc1e-499c-90d9-6560fe139ab5 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:18.421474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdd2cd-fc1e-499c-90d9-6560fe139ab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:18.421753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdd2cd-fc1e-499c-90d9-6560fe139ab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:18.430705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cdd2cd-fc1e-499c-90d9-6560fe139ab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:18.432138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cdd2cd-fc1e-499c-90d9-6560fe139ab5 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:18.441860 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1232/2468] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:18 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:18.591217 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734a15d-fefd-4089-bd3d-0875f5caab80 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:18.601062 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734a15d-fefd-4089-bd3d-0875f5caab80 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:18.604776 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad085302-4ed1-4b44-a8e0-0db41194c6c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:18.605412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad085302-4ed1-4b44-a8e0-0db41194c6c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:18.605693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad085302-4ed1-4b44-a8e0-0db41194c6c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:18.605971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad085302-4ed1-4b44-a8e0-0db41194c6c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:18.663124 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad085302-4ed1-4b44-a8e0-0db41194c6c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:18.664554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad085302-4ed1-4b44-a8e0-0db41194c6c6 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:18.673593 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1237/2469] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:18 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:20.101207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cdd2cd-fc1e-499c-90d9-6560fe139ab5 service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:20.102666 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-32cdd2cd-fc1e-499c-90d9-6560fe139ab5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:20.179045 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cdd2cd-fc1e-499c-90d9-6560fe139ab5 service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:20.183774 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302db6c5-f9d3-49c1-8ae9-6e901f6bd6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:20.184738 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302db6c5-f9d3-49c1-8ae9-6e901f6bd6b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:20.185022 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302db6c5-f9d3-49c1-8ae9-6e901f6bd6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:20.185300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302db6c5-f9d3-49c1-8ae9-6e901f6bd6b6 admin admin] PATH_INFO: `/v3/users/f8b0e66a61544ef192c0c1045081050e` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:20.187272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302db6c5-f9d3-49c1-8ae9-6e901f6bd6b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8b0e66a61544ef192c0c1045081050e)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:20.189086 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302db6c5-f9d3-49c1-8ae9-6e901f6bd6b6 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:20.203298 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:20.243310 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-302db6c5-f9d3-49c1-8ae9-6e901f6bd6b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8b0e66a61544ef192c0c1045081050e'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:20.255905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-302db6c5-f9d3-49c1-8ae9-6e901f6bd6b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8b0e66a61544ef192c0c1045081050e'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:20.263131 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-302db6c5-f9d3-49c1-8ae9-6e901f6bd6b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8b0e66a61544ef192c0c1045081050e'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:20.263816 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-302db6c5-f9d3-49c1-8ae9-6e901f6bd6b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8b0e66a61544ef192c0c1045081050e'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:20.272231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-302db6c5-f9d3-49c1-8ae9-6e901f6bd6b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8b0e66a61544ef192c0c1045081050e'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:20.278198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1233/2470] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:19:20 2022] DELETE /identity/v3/users/f8b0e66a61544ef192c0c1045081050e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:20.282905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925af4f-3cdf-470c-9c98-a3c93b440072 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:20.283384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925af4f-3cdf-470c-9c98-a3c93b440072 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:20.283482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925af4f-3cdf-470c-9c98-a3c93b440072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:20.299867 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-1925af4f-3cdf-470c-9c98-a3c93b440072 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:20.301073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-1925af4f-3cdf-470c-9c98-a3c93b440072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:20.360038 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1238/2471] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:20.369806 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302db6c5-f9d3-49c1-8ae9-6e901f6bd6b6 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:20.371242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-302db6c5-f9d3-49c1-8ae9-6e901f6bd6b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:20.428918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302db6c5-f9d3-49c1-8ae9-6e901f6bd6b6 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:20.432596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993f13a-e1c8-4d80-9925-2a0b1255997c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:20.433243 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993f13a-e1c8-4d80-9925-2a0b1255997c service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:20.433526 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993f13a-e1c8-4d80-9925-2a0b1255997c service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:20.433808 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993f13a-e1c8-4d80-9925-2a0b1255997c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:20.443571 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4993f13a-e1c8-4d80-9925-2a0b1255997c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:20.445004 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4993f13a-e1c8-4d80-9925-2a0b1255997c service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:20.454733 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1234/2472] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:20.729356 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925af4f-3cdf-470c-9c98-a3c93b440072 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:20.731027 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-1925af4f-3cdf-470c-9c98-a3c93b440072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:20.743746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925af4f-3cdf-470c-9c98-a3c93b440072 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:20.748432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7339fd-51f1-4841-b96b-cf6ad30f4286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:20.750431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7339fd-51f1-4841-b96b-cf6ad30f4286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:20.750768 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7339fd-51f1-4841-b96b-cf6ad30f4286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:20.751067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7339fd-51f1-4841-b96b-cf6ad30f4286 admin admin] PATH_INFO: `/v3/projects/d828bf5b18a64f78882739463a4ae7ca` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:20.752331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7339fd-51f1-4841-b96b-cf6ad30f4286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d828bf5b18a64f78882739463a4ae7ca)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:20.754235 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7339fd-51f1-4841-b96b-cf6ad30f4286 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:20.804233 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-db7339fd-51f1-4841-b96b-cf6ad30f4286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd828bf5b18a64f78882739463a4ae7ca', 'request_id': 'req-db7339fd-51f1-4841-b96b-cf6ad30f4286'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:20.809637 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-db7339fd-51f1-4841-b96b-cf6ad30f4286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd828bf5b18a64f78882739463a4ae7ca', 'request_id': 'req-db7339fd-51f1-4841-b96b-cf6ad30f4286'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:20.816546 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-db7339fd-51f1-4841-b96b-cf6ad30f4286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd828bf5b18a64f78882739463a4ae7ca', 'request_id': 'req-db7339fd-51f1-4841-b96b-cf6ad30f4286'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:20.829463 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-db7339fd-51f1-4841-b96b-cf6ad30f4286 admin admin] The token cache is being invalidate because project d828bf5b18a64f78882739463a4ae7ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:19:20.829749 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-db7339fd-51f1-4841-b96b-cf6ad30f4286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:20.830370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-db7339fd-51f1-4841-b96b-cf6ad30f4286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:20.831914 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1239/2473] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:19:20 2022] DELETE /identity/v3/projects/d828bf5b18a64f78882739463a4ae7ca => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:20.836938 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4d1a7-71d1-4c11-b049-51df36ca8447 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:20.837230 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4d1a7-71d1-4c11-b049-51df36ca8447 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:20.837511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4d1a7-71d1-4c11-b049-51df36ca8447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:20.855147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-bee4d1a7-71d1-4c11-b049-51df36ca8447 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:20.856350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-bee4d1a7-71d1-4c11-b049-51df36ca8447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:20.914891 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1235/2474] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:20.922765 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be09a3-1a4c-41fc-9cdf-4ecc2effa401 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:20.923058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be09a3-1a4c-41fc-9cdf-4ecc2effa401 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:20.923337 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be09a3-1a4c-41fc-9cdf-4ecc2effa401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:20.935616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:20.942609 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-b8be09a3-1a4c-41fc-9cdf-4ecc2effa401 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:20.944343 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b8be09a3-1a4c-41fc-9cdf-4ecc2effa401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:20.953888 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1240/2475] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:20.958014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4d1a7-71d1-4c11-b049-51df36ca8447 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:20.967534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4d1a7-71d1-4c11-b049-51df36ca8447 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:20.972219 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c382000-cf18-48ad-82d8-c3e1f8e5fdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:20.972872 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c382000-cf18-48ad-82d8-c3e1f8e5fdd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:20.973158 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c382000-cf18-48ad-82d8-c3e1f8e5fdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:20.973439 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c382000-cf18-48ad-82d8-c3e1f8e5fdd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:20.974168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c382000-cf18-48ad-82d8-c3e1f8e5fdd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:19:20.974480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c382000-cf18-48ad-82d8-c3e1f8e5fdd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:20.976158 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c382000-cf18-48ad-82d8-c3e1f8e5fdd9 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:20.987831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1236/2476] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:19:20.991864 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be09a3-1a4c-41fc-9cdf-4ecc2effa401 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:21.002248 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be09a3-1a4c-41fc-9cdf-4ecc2effa401 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:21.006841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa6225-d1eb-48b0-8b00-3a743330b3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:21.007596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa6225-d1eb-48b0-8b00-3a743330b3b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:21.007881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa6225-d1eb-48b0-8b00-3a743330b3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:21.008162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa6225-d1eb-48b0-8b00-3a743330b3b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:21.008855 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa6225-d1eb-48b0-8b00-3a743330b3b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:21.010740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa6225-d1eb-48b0-8b00-3a743330b3b1 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:21.036772 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1241/2477] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:19:20 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:19:21.040791 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c382000-cf18-48ad-82d8-c3e1f8e5fdd9 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:21.050563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c382000-cf18-48ad-82d8-c3e1f8e5fdd9 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:21.067664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:21.108993 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ac104-3f8c-4ac7-9972-b1ee8647dd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:21.109727 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ac104-3f8c-4ac7-9972-b1ee8647dd10 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:21.109926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ac104-3f8c-4ac7-9972-b1ee8647dd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:21.110203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ac104-3f8c-4ac7-9972-b1ee8647dd10 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:21.110912 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ac104-3f8c-4ac7-9972-b1ee8647dd10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:21.112821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ac104-3f8c-4ac7-9972-b1ee8647dd10 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:21.132023 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1237/2478] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:19:21 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:21.136180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa6225-d1eb-48b0-8b00-3a743330b3b1 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:21.146254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa6225-d1eb-48b0-8b00-3a743330b3b1 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:21.150788 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d9e90-ab19-4769-b425-f43b352071e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:21.151440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d9e90-ab19-4769-b425-f43b352071e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:21.151722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d9e90-ab19-4769-b425-f43b352071e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:21.151999 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d9e90-ab19-4769-b425-f43b352071e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:21.152721 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d9e90-ab19-4769-b425-f43b352071e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:21.154295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d9e90-ab19-4769-b425-f43b352071e0 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:21.165071 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1242/2479] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:21.168965 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ac104-3f8c-4ac7-9972-b1ee8647dd10 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:21.179747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ac104-3f8c-4ac7-9972-b1ee8647dd10 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:21.184778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a833f70-27b1-4675-a880-c770f86f1b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:21.185484 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a833f70-27b1-4675-a880-c770f86f1b91 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:21.185756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a833f70-27b1-4675-a880-c770f86f1b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:21.186031 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a833f70-27b1-4675-a880-c770f86f1b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:21.187525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a833f70-27b1-4675-a880-c770f86f1b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:21.189140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a833f70-27b1-4675-a880-c770f86f1b91 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:21.199464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1238/2480] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:21.204013 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d9e90-ab19-4769-b425-f43b352071e0 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:21.213893 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d9e90-ab19-4769-b425-f43b352071e0 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:21.218941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc272e78-ceaa-4bf6-add2-ea913823cec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:21.219814 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc272e78-ceaa-4bf6-add2-ea913823cec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:21.220096 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc272e78-ceaa-4bf6-add2-ea913823cec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:21.220369 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc272e78-ceaa-4bf6-add2-ea913823cec6 admin admin] PATH_INFO: `/v3/projects/e6a9958d940546b99e37b85acc95ad7c/users/bcc742eb9e7b42d9bb41f04bb58038db/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:21.235078 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc272e78-ceaa-4bf6-add2-ea913823cec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a9958d940546b99e37b85acc95ad7c, user_id=bcc742eb9e7b42d9bb41f04bb58038db, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:21.238045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc272e78-ceaa-4bf6-add2-ea913823cec6 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:21.258555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1243/2481] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:21 2022] PUT /identity/v3/projects/e6a9958d940546b99e37b85acc95ad7c/users/bcc742eb9e7b42d9bb41f04bb58038db/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:21.260463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54b295-16ed-4926-8d12-3c2fabcbcd00 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:21.260902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54b295-16ed-4926-8d12-3c2fabcbcd00 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:21.262089 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54b295-16ed-4926-8d12-3c2fabcbcd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:21.280239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-ea54b295-16ed-4926-8d12-3c2fabcbcd00 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:21.281431 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ea54b295-16ed-4926-8d12-3c2fabcbcd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:21.340898 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1239/2482] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:19:21.350351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc272e78-ceaa-4bf6-add2-ea913823cec6 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:21.352192 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-cc272e78-ceaa-4bf6-add2-ea913823cec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:21.409247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc272e78-ceaa-4bf6-add2-ea913823cec6 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:21.412992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea023a49-c0ca-4994-ba4d-f05be7c7c6e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:21.413645 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea023a49-c0ca-4994-ba4d-f05be7c7c6e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:21.413933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea023a49-c0ca-4994-ba4d-f05be7c7c6e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:21.414214 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea023a49-c0ca-4994-ba4d-f05be7c7c6e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:21.423931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea023a49-c0ca-4994-ba4d-f05be7c7c6e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:21.425356 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea023a49-c0ca-4994-ba4d-f05be7c7c6e4 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:21.434752 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1244/2483] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:25.644827 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8e594-8257-4f95-9a23-0630ad60f6a8 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:25.645723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8e594-8257-4f95-9a23-0630ad60f6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:25.646177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8e594-8257-4f95-9a23-0630ad60f6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:25.688631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-6ca8e594-8257-4f95-9a23-0630ad60f6a8 None None] MFA Rules not processed for user `bcc742eb9e7b42d9bb41f04bb58038db`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:25.690208 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6ca8e594-8257-4f95-9a23-0630ad60f6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:25.810511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1240/2484] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:25.812243 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea023a49-c0ca-4994-ba4d-f05be7c7c6e4 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:25.824976 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea023a49-c0ca-4994-ba4d-f05be7c7c6e4 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:25.837520 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8695bc-2bb6-49f7-b76b-3e503d8b4d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:25.838215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8695bc-2bb6-49f7-b76b-3e503d8b4d6a admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:25.838504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8695bc-2bb6-49f7-b76b-3e503d8b4d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:25.838833 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8695bc-2bb6-49f7-b76b-3e503d8b4d6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:25.840197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8695bc-2bb6-49f7-b76b-3e503d8b4d6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:25.842324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8695bc-2bb6-49f7-b76b-3e503d8b4d6a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:25.883278 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1245/2485] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:19:25 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:25.887219 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8e594-8257-4f95-9a23-0630ad60f6a8 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:25.900344 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8e594-8257-4f95-9a23-0630ad60f6a8 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:25.930507 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:25.968123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0233fce-97c9-4ac9-8c94-063d478fb7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:25.969118 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0233fce-97c9-4ac9-8c94-063d478fb7d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:25.969398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0233fce-97c9-4ac9-8c94-063d478fb7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:25.969738 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0233fce-97c9-4ac9-8c94-063d478fb7d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:25.970534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0233fce-97c9-4ac9-8c94-063d478fb7d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:25.972529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0233fce-97c9-4ac9-8c94-063d478fb7d5 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:25.994841 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1241/2486] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:19:25 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:25.999594 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8695bc-2bb6-49f7-b76b-3e503d8b4d6a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:26.011724 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8695bc-2bb6-49f7-b76b-3e503d8b4d6a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:26.019331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d0846-9fb7-4114-80f3-766a052cbd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:26.020574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d0846-9fb7-4114-80f3-766a052cbd05 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:26.021107 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d0846-9fb7-4114-80f3-766a052cbd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:26.021940 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d0846-9fb7-4114-80f3-766a052cbd05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:26.022887 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d0846-9fb7-4114-80f3-766a052cbd05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:26.024620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d0846-9fb7-4114-80f3-766a052cbd05 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:26.041195 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1246/2487] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:19:26.045146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0233fce-97c9-4ac9-8c94-063d478fb7d5 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:26.056946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0233fce-97c9-4ac9-8c94-063d478fb7d5 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:26.072169 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e61d7-b771-44dc-b819-b8591faeedfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:26.073111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e61d7-b771-44dc-b819-b8591faeedfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:26.073402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e61d7-b771-44dc-b819-b8591faeedfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:26.073685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e61d7-b771-44dc-b819-b8591faeedfb admin admin] PATH_INFO: `/v3/projects/ab5e0c8af01e445eb73ffb61ed0fef66/users/9386ad7adc354126b5701dda57f2def5/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:26.109029 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e61d7-b771-44dc-b819-b8591faeedfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab5e0c8af01e445eb73ffb61ed0fef66, user_id=9386ad7adc354126b5701dda57f2def5, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:26.112208 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e61d7-b771-44dc-b819-b8591faeedfb admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:26.131990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1242/2488] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:26 2022] PUT /identity/v3/projects/ab5e0c8af01e445eb73ffb61ed0fef66/users/9386ad7adc354126b5701dda57f2def5/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:26.135781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d0846-9fb7-4114-80f3-766a052cbd05 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:26.148244 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d0846-9fb7-4114-80f3-766a052cbd05 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:26.226083 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a8e69-aa22-4ad9-b592-55d1dfccfa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:26.226809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a8e69-aa22-4ad9-b592-55d1dfccfa04 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:26.227096 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a8e69-aa22-4ad9-b592-55d1dfccfa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:26.227374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a8e69-aa22-4ad9-b592-55d1dfccfa04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:26.228109 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a8e69-aa22-4ad9-b592-55d1dfccfa04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:26.230321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a8e69-aa22-4ad9-b592-55d1dfccfa04 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:26.247616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1247/2489] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:26.252756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e61d7-b771-44dc-b819-b8591faeedfb admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:26.265853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e61d7-b771-44dc-b819-b8591faeedfb admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:26.272876 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600095c-d57e-436c-adc4-37a458ea07e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:26.274140 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600095c-d57e-436c-adc4-37a458ea07e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:26.274472 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600095c-d57e-436c-adc4-37a458ea07e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:26.274899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600095c-d57e-436c-adc4-37a458ea07e7 admin admin] PATH_INFO: `/v3/projects/ab5e0c8af01e445eb73ffb61ed0fef66/users/9386ad7adc354126b5701dda57f2def5/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:26.277300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600095c-d57e-436c-adc4-37a458ea07e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab5e0c8af01e445eb73ffb61ed0fef66, user_id=9386ad7adc354126b5701dda57f2def5, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:26.280664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600095c-d57e-436c-adc4-37a458ea07e7 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:26.294596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1243/2490] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:26 2022] PUT /identity/v3/projects/ab5e0c8af01e445eb73ffb61ed0fef66/users/9386ad7adc354126b5701dda57f2def5/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:26.324351 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a8e69-aa22-4ad9-b592-55d1dfccfa04 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:26.326831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6a5a8e69-aa22-4ad9-b592-55d1dfccfa04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:26.340222 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a8e69-aa22-4ad9-b592-55d1dfccfa04 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:26.345494 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:26.417549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a490681-2962-48fd-b5b4-fc68acbb8055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:26.418486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a490681-2962-48fd-b5b4-fc68acbb8055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:26.418809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a490681-2962-48fd-b5b4-fc68acbb8055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:26.419091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a490681-2962-48fd-b5b4-fc68acbb8055 admin admin] PATH_INFO: `/v3/users/5edb42d224d643a9912f06fcf7c598c0` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:26.420405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a490681-2962-48fd-b5b4-fc68acbb8055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5edb42d224d643a9912f06fcf7c598c0)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:26.423464 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a490681-2962-48fd-b5b4-fc68acbb8055 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:26.474634 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-9a490681-2962-48fd-b5b4-fc68acbb8055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5edb42d224d643a9912f06fcf7c598c0'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:26.490567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-9a490681-2962-48fd-b5b4-fc68acbb8055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5edb42d224d643a9912f06fcf7c598c0'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:26.498948 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-9a490681-2962-48fd-b5b4-fc68acbb8055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5edb42d224d643a9912f06fcf7c598c0'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:26.499946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-9a490681-2962-48fd-b5b4-fc68acbb8055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5edb42d224d643a9912f06fcf7c598c0'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:26.507427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-9a490681-2962-48fd-b5b4-fc68acbb8055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5edb42d224d643a9912f06fcf7c598c0'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:26.513881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1248/2491] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:19:26 2022] DELETE /identity/v3/users/5edb42d224d643a9912f06fcf7c598c0 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:26.518587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d694cee-b4b8-4316-bed5-12669edbe057 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:26.518930 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d694cee-b4b8-4316-bed5-12669edbe057 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:26.519202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d694cee-b4b8-4316-bed5-12669edbe057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:26.538375 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-3d694cee-b4b8-4316-bed5-12669edbe057 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:26.539291 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3d694cee-b4b8-4316-bed5-12669edbe057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:26.616547 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1244/2492] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:26.632770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a490681-2962-48fd-b5b4-fc68acbb8055 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:26.636123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9a490681-2962-48fd-b5b4-fc68acbb8055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:26.718296 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a490681-2962-48fd-b5b4-fc68acbb8055 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:26.725448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a0531-8088-4c73-91ab-7584c15821da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:26.726591 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a0531-8088-4c73-91ab-7584c15821da service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:26.726914 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a0531-8088-4c73-91ab-7584c15821da service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:26.727392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a0531-8088-4c73-91ab-7584c15821da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:26.740622 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a0531-8088-4c73-91ab-7584c15821da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:26.742658 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a0531-8088-4c73-91ab-7584c15821da service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:26.757750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1249/2493] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:27.529674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d694cee-b4b8-4316-bed5-12669edbe057 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:27.531248 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3d694cee-b4b8-4316-bed5-12669edbe057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:27.544216 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d694cee-b4b8-4316-bed5-12669edbe057 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:27.557235 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba78ca0-17ba-4852-a09f-423ac664de79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:27.558335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba78ca0-17ba-4852-a09f-423ac664de79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:27.558513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba78ca0-17ba-4852-a09f-423ac664de79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:27.558778 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba78ca0-17ba-4852-a09f-423ac664de79 admin admin] PATH_INFO: `/v3/projects/eb75cea8b9a14bc1915bb0a43e07a30e` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:27.560295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba78ca0-17ba-4852-a09f-423ac664de79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb75cea8b9a14bc1915bb0a43e07a30e)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:27.563267 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba78ca0-17ba-4852-a09f-423ac664de79 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:27.621940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4ba78ca0-17ba-4852-a09f-423ac664de79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb75cea8b9a14bc1915bb0a43e07a30e', 'request_id': 'req-4ba78ca0-17ba-4852-a09f-423ac664de79'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:27.630933 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4ba78ca0-17ba-4852-a09f-423ac664de79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb75cea8b9a14bc1915bb0a43e07a30e', 'request_id': 'req-4ba78ca0-17ba-4852-a09f-423ac664de79'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:27.641377 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4ba78ca0-17ba-4852-a09f-423ac664de79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb75cea8b9a14bc1915bb0a43e07a30e', 'request_id': 'req-4ba78ca0-17ba-4852-a09f-423ac664de79'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:27.656088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4ba78ca0-17ba-4852-a09f-423ac664de79 admin admin] The token cache is being invalidate because project eb75cea8b9a14bc1915bb0a43e07a30e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:19:27.656846 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4ba78ca0-17ba-4852-a09f-423ac664de79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:27.657917 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4ba78ca0-17ba-4852-a09f-423ac664de79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:27.661309 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1245/2494] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:19:27 2022] DELETE /identity/v3/projects/eb75cea8b9a14bc1915bb0a43e07a30e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:19:27.665551 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a0531-8088-4c73-91ab-7584c15821da service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:27.678617 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a0531-8088-4c73-91ab-7584c15821da service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:27.720593 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:27.779529 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af273f-1378-46ca-a405-b88b8b8060dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:27.780893 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af273f-1378-46ca-a405-b88b8b8060dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:27.781326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af273f-1378-46ca-a405-b88b8b8060dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:27.781741 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af273f-1378-46ca-a405-b88b8b8060dd admin admin] PATH_INFO: `/v3/users/ee31547c783f41839bb0b683781ff326` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:27.783380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af273f-1378-46ca-a405-b88b8b8060dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee31547c783f41839bb0b683781ff326)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:27.786101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af273f-1378-46ca-a405-b88b8b8060dd admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:27.865927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-95af273f-1378-46ca-a405-b88b8b8060dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee31547c783f41839bb0b683781ff326'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:27.872364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-95af273f-1378-46ca-a405-b88b8b8060dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee31547c783f41839bb0b683781ff326'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:27.880270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-95af273f-1378-46ca-a405-b88b8b8060dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee31547c783f41839bb0b683781ff326'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:27.881014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-95af273f-1378-46ca-a405-b88b8b8060dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee31547c783f41839bb0b683781ff326'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:27.889102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-95af273f-1378-46ca-a405-b88b8b8060dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee31547c783f41839bb0b683781ff326'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:27.896546 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1250/2495] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:19:27 2022] DELETE /identity/v3/users/ee31547c783f41839bb0b683781ff326 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:28.210295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba78ca0-17ba-4852-a09f-423ac664de79 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:28.212427 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4ba78ca0-17ba-4852-a09f-423ac664de79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:28.231074 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:28.234776 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba78ca0-17ba-4852-a09f-423ac664de79 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:28.313339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f91f8-8ac3-4a5c-89c9-11059482577f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:28.314271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f91f8-8ac3-4a5c-89c9-11059482577f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:28.314604 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f91f8-8ac3-4a5c-89c9-11059482577f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:28.314914 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f91f8-8ac3-4a5c-89c9-11059482577f admin admin] PATH_INFO: `/v3/projects/d32dd1dc692245deaf04a07ed0f3bb93` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:28.316605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f91f8-8ac3-4a5c-89c9-11059482577f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d32dd1dc692245deaf04a07ed0f3bb93)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:28.318523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f91f8-8ac3-4a5c-89c9-11059482577f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:28.391539 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-724f91f8-8ac3-4a5c-89c9-11059482577f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd32dd1dc692245deaf04a07ed0f3bb93', 'request_id': 'req-724f91f8-8ac3-4a5c-89c9-11059482577f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:28.398468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-724f91f8-8ac3-4a5c-89c9-11059482577f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd32dd1dc692245deaf04a07ed0f3bb93', 'request_id': 'req-724f91f8-8ac3-4a5c-89c9-11059482577f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:28.407311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-724f91f8-8ac3-4a5c-89c9-11059482577f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd32dd1dc692245deaf04a07ed0f3bb93', 'request_id': 'req-724f91f8-8ac3-4a5c-89c9-11059482577f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:28.423657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-724f91f8-8ac3-4a5c-89c9-11059482577f admin admin] The token cache is being invalidate because project d32dd1dc692245deaf04a07ed0f3bb93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:19:28.423971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-724f91f8-8ac3-4a5c-89c9-11059482577f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:28.424803 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-724f91f8-8ac3-4a5c-89c9-11059482577f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:28.426642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1246/2496] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:19:28 2022] DELETE /identity/v3/projects/d32dd1dc692245deaf04a07ed0f3bb93 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:28.431366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b367d4e-a368-4422-9617-f4c026ea0f51 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:28.431664 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b367d4e-a368-4422-9617-f4c026ea0f51 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:28.432085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b367d4e-a368-4422-9617-f4c026ea0f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:28.451228 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-4b367d4e-a368-4422-9617-f4c026ea0f51 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:28.452738 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-4b367d4e-a368-4422-9617-f4c026ea0f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:28.525704 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1251/2497] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:28.533557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0fa5b-c7a2-4895-b326-b7524941c39d None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:28.534509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0fa5b-c7a2-4895-b326-b7524941c39d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:28.535055 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0fa5b-c7a2-4895-b326-b7524941c39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:28.568102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-9be0fa5b-c7a2-4895-b326-b7524941c39d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:28.569468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-9be0fa5b-c7a2-4895-b326-b7524941c39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:28.580475 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1247/2498] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:28.586140 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b367d4e-a368-4422-9617-f4c026ea0f51 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:28.597030 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b367d4e-a368-4422-9617-f4c026ea0f51 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:28.602213 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1dae0d-f6f7-4577-96f7-bf941cc38e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:28.602895 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dae0d-f6f7-4577-96f7-bf941cc38e6e admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:28.603236 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dae0d-f6f7-4577-96f7-bf941cc38e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:28.603887 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dae0d-f6f7-4577-96f7-bf941cc38e6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:28.605321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1dae0d-f6f7-4577-96f7-bf941cc38e6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:19:28.605321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1dae0d-f6f7-4577-96f7-bf941cc38e6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:28.606392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1dae0d-f6f7-4577-96f7-bf941cc38e6e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:28.621278 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1252/2499] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:19:28.626130 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0fa5b-c7a2-4895-b326-b7524941c39d None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:28.635537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0fa5b-c7a2-4895-b326-b7524941c39d None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:28.640302 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4f8c0-5113-4a8b-a873-ea1ef17e42fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:28.640975 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4f8c0-5113-4a8b-a873-ea1ef17e42fa admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:28.641278 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4f8c0-5113-4a8b-a873-ea1ef17e42fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:28.641557 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4f8c0-5113-4a8b-a873-ea1ef17e42fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:28.642253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4f8c0-5113-4a8b-a873-ea1ef17e42fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:28.644187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4f8c0-5113-4a8b-a873-ea1ef17e42fa admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:28.674671 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1248/2500] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:19:28 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:28.679439 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1dae0d-f6f7-4577-96f7-bf941cc38e6e admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:28.692411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1dae0d-f6f7-4577-96f7-bf941cc38e6e admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:28.734844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:28.766946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f18f4-eef2-497f-b821-67686393d9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:28.767639 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f18f4-eef2-497f-b821-67686393d9fd admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:28.767923 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f18f4-eef2-497f-b821-67686393d9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:28.768199 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f18f4-eef2-497f-b821-67686393d9fd admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:28.768932 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f18f4-eef2-497f-b821-67686393d9fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:28.770881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f18f4-eef2-497f-b821-67686393d9fd admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:28.795212 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1253/2501] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:19:28 2022] POST /identity/v3/users => generated 290 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:28.796433 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4f8c0-5113-4a8b-a873-ea1ef17e42fa admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:28.809218 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4f8c0-5113-4a8b-a873-ea1ef17e42fa admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:28.816446 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4485a-b69c-468c-9bfd-d310324a80d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:28.817461 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4485a-b69c-468c-9bfd-d310324a80d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:28.818085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4485a-b69c-468c-9bfd-d310324a80d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:28.820619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4485a-b69c-468c-9bfd-d310324a80d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:28.821702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4485a-b69c-468c-9bfd-d310324a80d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:28.823675 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4485a-b69c-468c-9bfd-d310324a80d1 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:28.836980 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1249/2502] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:28.840823 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f18f4-eef2-497f-b821-67686393d9fd admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:28.860917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f18f4-eef2-497f-b821-67686393d9fd admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:28.866609 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51c1c5-dcb1-40ad-a65a-c56eea744c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:28.869225 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51c1c5-dcb1-40ad-a65a-c56eea744c95 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:28.870365 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51c1c5-dcb1-40ad-a65a-c56eea744c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:28.870964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51c1c5-dcb1-40ad-a65a-c56eea744c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:28.872144 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51c1c5-dcb1-40ad-a65a-c56eea744c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:28.875468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51c1c5-dcb1-40ad-a65a-c56eea744c95 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:28.902732 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1254/2503] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:28.907136 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4485a-b69c-468c-9bfd-d310324a80d1 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:28.924678 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4485a-b69c-468c-9bfd-d310324a80d1 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:28.930567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4535b86-6b86-47b3-8d1e-8564cc6053a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:28.931886 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4535b86-6b86-47b3-8d1e-8564cc6053a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:28.932397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4535b86-6b86-47b3-8d1e-8564cc6053a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:28.932910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4535b86-6b86-47b3-8d1e-8564cc6053a0 admin admin] PATH_INFO: `/v3/projects/8b3d9dfa1c2f4b75923b5e6fbc4498e3/users/b29ec72f8f5d4ab798a5d8826bda29d2/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:28.960749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4535b86-6b86-47b3-8d1e-8564cc6053a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b3d9dfa1c2f4b75923b5e6fbc4498e3, user_id=b29ec72f8f5d4ab798a5d8826bda29d2, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:28.976401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4535b86-6b86-47b3-8d1e-8564cc6053a0 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:29.001896 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1250/2504] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:28 2022] PUT /identity/v3/projects/8b3d9dfa1c2f4b75923b5e6fbc4498e3/users/b29ec72f8f5d4ab798a5d8826bda29d2/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:29.013846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee6257-93fc-4dc8-b0d2-36aa91c7c6c4 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:29.014909 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee6257-93fc-4dc8-b0d2-36aa91c7c6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:29.015274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee6257-93fc-4dc8-b0d2-36aa91c7c6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:29.059555 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-98ee6257-93fc-4dc8-b0d2-36aa91c7c6c4 None None] MFA Rules not processed for user `b29ec72f8f5d4ab798a5d8826bda29d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:29.064651 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-98ee6257-93fc-4dc8-b0d2-36aa91c7c6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:29.237997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1255/2505] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:29.243304 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4535b86-6b86-47b3-8d1e-8564cc6053a0 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:29.334148 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:29.383664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4535b86-6b86-47b3-8d1e-8564cc6053a0 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:29.402781 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0fd81-49cf-47a9-a7a7-a946ba31697d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:29.404272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0fd81-49cf-47a9-a7a7-a946ba31697d admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:29.405429 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0fd81-49cf-47a9-a7a7-a946ba31697d admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:29.406708 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da0fd81-49cf-47a9-a7a7-a946ba31697d admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:29.407714 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da0fd81-49cf-47a9-a7a7-a946ba31697d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:29.411012 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da0fd81-49cf-47a9-a7a7-a946ba31697d admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:29.481800 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1251/2506] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:19:29 2022] POST /identity/v3/projects => generated 377 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:29.486589 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee6257-93fc-4dc8-b0d2-36aa91c7c6c4 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:29.504091 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee6257-93fc-4dc8-b0d2-36aa91c7c6c4 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:29.603102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa93d16-1010-4718-bad1-b9e147ca6f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:29.604417 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa93d16-1010-4718-bad1-b9e147ca6f14 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:29.604809 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa93d16-1010-4718-bad1-b9e147ca6f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:29.605417 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa93d16-1010-4718-bad1-b9e147ca6f14 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:29.611100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa93d16-1010-4718-bad1-b9e147ca6f14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:29.616138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa93d16-1010-4718-bad1-b9e147ca6f14 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:29.648231 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1256/2507] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:19:29 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:29.651766 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0fd81-49cf-47a9-a7a7-a946ba31697d admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:29.669398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da0fd81-49cf-47a9-a7a7-a946ba31697d admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:29.683654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66681fa-eb5b-4a41-a22a-e124e6af3c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:29.684368 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66681fa-eb5b-4a41-a22a-e124e6af3c79 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:29.684657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66681fa-eb5b-4a41-a22a-e124e6af3c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:29.684941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66681fa-eb5b-4a41-a22a-e124e6af3c79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:29.691067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66681fa-eb5b-4a41-a22a-e124e6af3c79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:29.693717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66681fa-eb5b-4a41-a22a-e124e6af3c79 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:29.709593 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1252/2508] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:29.713414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa93d16-1010-4718-bad1-b9e147ca6f14 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:29.732384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa93d16-1010-4718-bad1-b9e147ca6f14 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:29.738857 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032a7be-ad15-43f9-b1da-dcca79711438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:29.739466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032a7be-ad15-43f9-b1da-dcca79711438 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:29.739754 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032a7be-ad15-43f9-b1da-dcca79711438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:29.740036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032a7be-ad15-43f9-b1da-dcca79711438 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:29.740784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032a7be-ad15-43f9-b1da-dcca79711438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:29.742409 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032a7be-ad15-43f9-b1da-dcca79711438 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:29.755196 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1257/2509] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:29.759032 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66681fa-eb5b-4a41-a22a-e124e6af3c79 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:29.774962 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66681fa-eb5b-4a41-a22a-e124e6af3c79 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:29.792700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa84ab5-4970-4e74-8792-4047d13c9ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:29.793767 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa84ab5-4970-4e74-8792-4047d13c9ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:29.794978 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa84ab5-4970-4e74-8792-4047d13c9ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:29.796543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa84ab5-4970-4e74-8792-4047d13c9ff7 admin admin] PATH_INFO: `/v3/projects/28584fe80f8340cf8958b66d3aa625fa/users/c07bff8c2c32480583989f2a7606395d/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:29.837890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa84ab5-4970-4e74-8792-4047d13c9ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28584fe80f8340cf8958b66d3aa625fa, user_id=c07bff8c2c32480583989f2a7606395d, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:29.843395 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa84ab5-4970-4e74-8792-4047d13c9ff7 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:29.885124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1253/2510] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:29 2022] PUT /identity/v3/projects/28584fe80f8340cf8958b66d3aa625fa/users/c07bff8c2c32480583989f2a7606395d/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:19:29.896064 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58e4a4-23da-4dd1-beae-48da8dd33fd7 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:29.896677 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58e4a4-23da-4dd1-beae-48da8dd33fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:29.896982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58e4a4-23da-4dd1-beae-48da8dd33fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:30.025572 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-cd58e4a4-23da-4dd1-beae-48da8dd33fd7 None None] MFA Rules not processed for user `c07bff8c2c32480583989f2a7606395d`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:30.029697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-cd58e4a4-23da-4dd1-beae-48da8dd33fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:30.110735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:30.260982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1258/2511] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:30.267295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa84ab5-4970-4e74-8792-4047d13c9ff7 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:30.417325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa84ab5-4970-4e74-8792-4047d13c9ff7 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:30.437856 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a25433-308a-490b-8cdd-4a3cbf325439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:30.438870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a25433-308a-490b-8cdd-4a3cbf325439 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:30.438870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a25433-308a-490b-8cdd-4a3cbf325439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:30.439122 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a25433-308a-490b-8cdd-4a3cbf325439 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:30.441907 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a25433-308a-490b-8cdd-4a3cbf325439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:30.448053 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a25433-308a-490b-8cdd-4a3cbf325439 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:30.499262 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1254/2512] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:19:30 2022] POST /identity/v3/projects => generated 379 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:30.503157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58e4a4-23da-4dd1-beae-48da8dd33fd7 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:30.520100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58e4a4-23da-4dd1-beae-48da8dd33fd7 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:30.621133 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f5bbd-5957-48e9-8996-667893f4a9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:30.622052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f5bbd-5957-48e9-8996-667893f4a9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:30.622824 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f5bbd-5957-48e9-8996-667893f4a9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:30.623241 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f5bbd-5957-48e9-8996-667893f4a9c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:30.624298 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f5bbd-5957-48e9-8996-667893f4a9c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:30.628402 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f5bbd-5957-48e9-8996-667893f4a9c2 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:30.662986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1259/2513] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:19:30 2022] POST /identity/v3/users => generated 296 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:19:30.668603 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a25433-308a-490b-8cdd-4a3cbf325439 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:30.688465 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a25433-308a-490b-8cdd-4a3cbf325439 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:30.695868 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d690da-6947-4d01-9247-f19f7bfcaffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:30.696887 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d690da-6947-4d01-9247-f19f7bfcaffc admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:30.697420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d690da-6947-4d01-9247-f19f7bfcaffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:30.697935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d690da-6947-4d01-9247-f19f7bfcaffc admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:30.699132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d690da-6947-4d01-9247-f19f7bfcaffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:30.701353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d690da-6947-4d01-9247-f19f7bfcaffc admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:30.727954 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1255/2514] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:30.729382 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f5bbd-5957-48e9-8996-667893f4a9c2 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:30.747199 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f5bbd-5957-48e9-8996-667893f4a9c2 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:30.759548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99383a-523d-4381-9ef6-6bd1852eada4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:30.760836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99383a-523d-4381-9ef6-6bd1852eada4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:30.761888 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99383a-523d-4381-9ef6-6bd1852eada4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:30.762282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99383a-523d-4381-9ef6-6bd1852eada4 admin admin] PATH_INFO: `/v3/projects/4e8a00b1f2db4cd39f005239c362f9a6/users/7d25e8b831fc436da755fdff357e9d1c/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:30.798758 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99383a-523d-4381-9ef6-6bd1852eada4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8a00b1f2db4cd39f005239c362f9a6, user_id=7d25e8b831fc436da755fdff357e9d1c, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:30.801713 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99383a-523d-4381-9ef6-6bd1852eada4 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:30.821793 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1260/2515] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:30 2022] PUT /identity/v3/projects/4e8a00b1f2db4cd39f005239c362f9a6/users/7d25e8b831fc436da755fdff357e9d1c/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:30.825757 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d690da-6947-4d01-9247-f19f7bfcaffc admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:30.845271 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d690da-6947-4d01-9247-f19f7bfcaffc admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:30.935635 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:30.958149 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f319b3c-2286-4223-8e90-4ab9fa9acda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:30.959285 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f319b3c-2286-4223-8e90-4ab9fa9acda1 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:30.959794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f319b3c-2286-4223-8e90-4ab9fa9acda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:30.960294 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f319b3c-2286-4223-8e90-4ab9fa9acda1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:30.961577 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f319b3c-2286-4223-8e90-4ab9fa9acda1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:30.963528 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f319b3c-2286-4223-8e90-4ab9fa9acda1 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:30.979504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1256/2516] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:30.984059 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99383a-523d-4381-9ef6-6bd1852eada4 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:30.996025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99383a-523d-4381-9ef6-6bd1852eada4 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:31.007178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f30eb-3d09-4bb7-8900-8a21bd49cecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:31.008190 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f30eb-3d09-4bb7-8900-8a21bd49cecb admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:31.008628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f30eb-3d09-4bb7-8900-8a21bd49cecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:31.009035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f30eb-3d09-4bb7-8900-8a21bd49cecb admin admin] PATH_INFO: `/v3/projects/4e8a00b1f2db4cd39f005239c362f9a6/users/7d25e8b831fc436da755fdff357e9d1c/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:31.016309 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f30eb-3d09-4bb7-8900-8a21bd49cecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8a00b1f2db4cd39f005239c362f9a6, user_id=7d25e8b831fc436da755fdff357e9d1c, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:31.019169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f30eb-3d09-4bb7-8900-8a21bd49cecb admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:31.042596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1261/2517] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:30 2022] PUT /identity/v3/projects/4e8a00b1f2db4cd39f005239c362f9a6/users/7d25e8b831fc436da755fdff357e9d1c/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:31.054644 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c081e-6c41-4567-8c9b-dcdfdbf0fb55 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:31.055042 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c081e-6c41-4567-8c9b-dcdfdbf0fb55 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:31.055317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c081e-6c41-4567-8c9b-dcdfdbf0fb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:31.089979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-947c081e-6c41-4567-8c9b-dcdfdbf0fb55 None None] MFA Rules not processed for user `7d25e8b831fc436da755fdff357e9d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:31.091481 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-947c081e-6c41-4567-8c9b-dcdfdbf0fb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:31.289443 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f30eb-3d09-4bb7-8900-8a21bd49cecb admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:31.289888 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1257/2518] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:31.291475 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-970f30eb-3d09-4bb7-8900-8a21bd49cecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:31.396858 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f30eb-3d09-4bb7-8900-8a21bd49cecb admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:31.402303 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c94c599-d26c-41d4-8c38-3d5558964bee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:31.403260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94c599-d26c-41d4-8c38-3d5558964bee service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:31.403545 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94c599-d26c-41d4-8c38-3d5558964bee service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:31.403830 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c94c599-d26c-41d4-8c38-3d5558964bee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:31.427067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c94c599-d26c-41d4-8c38-3d5558964bee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:31.428721 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c94c599-d26c-41d4-8c38-3d5558964bee service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:31.451732 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1262/2519] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:31.512161 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c081e-6c41-4567-8c9b-dcdfdbf0fb55 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:31.513692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-947c081e-6c41-4567-8c9b-dcdfdbf0fb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:31.557238 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c081e-6c41-4567-8c9b-dcdfdbf0fb55 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:31.569929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04139cc3-dae8-4763-9a9c-e5c70450dc75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:31.570646 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04139cc3-dae8-4763-9a9c-e5c70450dc75 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:31.571069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04139cc3-dae8-4763-9a9c-e5c70450dc75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:31.571548 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04139cc3-dae8-4763-9a9c-e5c70450dc75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:31.679588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04139cc3-dae8-4763-9a9c-e5c70450dc75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:31.681069 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04139cc3-dae8-4763-9a9c-e5c70450dc75 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:31.698901 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1258/2520] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:32.406425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df4cf5-227d-452c-b6ac-cdc69d7edae3 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:32.406905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df4cf5-227d-452c-b6ac-cdc69d7edae3 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:32.407687 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df4cf5-227d-452c-b6ac-cdc69d7edae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:32.429729 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:32.448102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-b9df4cf5-227d-452c-b6ac-cdc69d7edae3 None None] MFA Rules not processed for user `9386ad7adc354126b5701dda57f2def5`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:32.450843 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b9df4cf5-227d-452c-b6ac-cdc69d7edae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:32.558236 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1263/2521] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:32.572511 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04139cc3-dae8-4763-9a9c-e5c70450dc75 service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:32.574228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-04139cc3-dae8-4763-9a9c-e5c70450dc75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:32.671462 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04139cc3-dae8-4763-9a9c-e5c70450dc75 service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:32.681273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4c01c-a88d-431e-8ab0-1331acbabace service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:32.682111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4c01c-a88d-431e-8ab0-1331acbabace service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:32.682490 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4c01c-a88d-431e-8ab0-1331acbabace service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:32.683011 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4c01c-a88d-431e-8ab0-1331acbabace service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:32.685132 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-2cf4c01c-a88d-431e-8ab0-1331acbabace service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:32.697742 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:32.763929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4c01c-a88d-431e-8ab0-1331acbabace service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:32.766501 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4c01c-a88d-431e-8ab0-1331acbabace service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:32.779582 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1259/2522] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:33.927652 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefe426-ef4e-46ef-86c3-3f4c4ac11e52 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:33.928169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefe426-ef4e-46ef-86c3-3f4c4ac11e52 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:33.928361 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefe426-ef4e-46ef-86c3-3f4c4ac11e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:33.948406 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-0aefe426-ef4e-46ef-86c3-3f4c4ac11e52 None None] MFA Rules not processed for user `bcc742eb9e7b42d9bb41f04bb58038db`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:33.950796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-0aefe426-ef4e-46ef-86c3-3f4c4ac11e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:33.958021 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1264/2523] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:34.931435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4c01c-a88d-431e-8ab0-1331acbabace service nova] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:35.024875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4c01c-a88d-431e-8ab0-1331acbabace service nova] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:35.033350 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1a10a-325e-43db-bf0f-a3c97c3f5c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:35.034269 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1a10a-325e-43db-bf0f-a3c97c3f5c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:35.034546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1a10a-325e-43db-bf0f-a3c97c3f5c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:35.034853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1a10a-325e-43db-bf0f-a3c97c3f5c6a admin admin] PATH_INFO: `/v3/users/b29ec72f8f5d4ab798a5d8826bda29d2` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:35.036111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1a10a-325e-43db-bf0f-a3c97c3f5c6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b29ec72f8f5d4ab798a5d8826bda29d2)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:35.038039 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1a10a-325e-43db-bf0f-a3c97c3f5c6a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:35.096837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-82d1a10a-325e-43db-bf0f-a3c97c3f5c6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b29ec72f8f5d4ab798a5d8826bda29d2'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:35.109906 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-82d1a10a-325e-43db-bf0f-a3c97c3f5c6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b29ec72f8f5d4ab798a5d8826bda29d2'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:35.117299 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-82d1a10a-325e-43db-bf0f-a3c97c3f5c6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b29ec72f8f5d4ab798a5d8826bda29d2'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:35.118099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-82d1a10a-325e-43db-bf0f-a3c97c3f5c6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b29ec72f8f5d4ab798a5d8826bda29d2'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:35.126196 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-82d1a10a-325e-43db-bf0f-a3c97c3f5c6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b29ec72f8f5d4ab798a5d8826bda29d2'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:35.133685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1260/2524] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:19:34 2022] DELETE /identity/v3/users/b29ec72f8f5d4ab798a5d8826bda29d2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:35.138458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f7c7d-19a2-49ff-b11f-78727262af73 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:35.139533 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f7c7d-19a2-49ff-b11f-78727262af73 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:35.139642 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f7c7d-19a2-49ff-b11f-78727262af73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:35.158109 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-1d3f7c7d-19a2-49ff-b11f-78727262af73 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:35.159298 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-1d3f7c7d-19a2-49ff-b11f-78727262af73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:35.228643 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1265/2525] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:19:35.240016 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1a10a-325e-43db-bf0f-a3c97c3f5c6a admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:35.241478 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-82d1a10a-325e-43db-bf0f-a3c97c3f5c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:35.283522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:35.311952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1a10a-325e-43db-bf0f-a3c97c3f5c6a admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:35.315841 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07355f-cbb2-4c78-831a-6fe97a74eb3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:35.316573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07355f-cbb2-4c78-831a-6fe97a74eb3f service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:35.316785 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07355f-cbb2-4c78-831a-6fe97a74eb3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:35.317066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07355f-cbb2-4c78-831a-6fe97a74eb3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:35.327548 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07355f-cbb2-4c78-831a-6fe97a74eb3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:35.329209 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07355f-cbb2-4c78-831a-6fe97a74eb3f service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:35.341503 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1261/2526] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:35.756463 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f7c7d-19a2-49ff-b11f-78727262af73 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:35.758570 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-1d3f7c7d-19a2-49ff-b11f-78727262af73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:35.774028 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f7c7d-19a2-49ff-b11f-78727262af73 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:35.782508 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e41715-7597-49af-91e9-dc7a9054603e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:35.783788 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41715-7597-49af-91e9-dc7a9054603e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:35.784554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41715-7597-49af-91e9-dc7a9054603e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:35.785303 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41715-7597-49af-91e9-dc7a9054603e admin admin] PATH_INFO: `/v3/projects/8b3d9dfa1c2f4b75923b5e6fbc4498e3` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:35.788817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e41715-7597-49af-91e9-dc7a9054603e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b3d9dfa1c2f4b75923b5e6fbc4498e3)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:35.793713 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e41715-7597-49af-91e9-dc7a9054603e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:35.898938 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-86e41715-7597-49af-91e9-dc7a9054603e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b3d9dfa1c2f4b75923b5e6fbc4498e3', 'request_id': 'req-86e41715-7597-49af-91e9-dc7a9054603e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:35.914473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-86e41715-7597-49af-91e9-dc7a9054603e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b3d9dfa1c2f4b75923b5e6fbc4498e3', 'request_id': 'req-86e41715-7597-49af-91e9-dc7a9054603e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:35.923184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-86e41715-7597-49af-91e9-dc7a9054603e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b3d9dfa1c2f4b75923b5e6fbc4498e3', 'request_id': 'req-86e41715-7597-49af-91e9-dc7a9054603e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:35.944506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-86e41715-7597-49af-91e9-dc7a9054603e admin admin] The token cache is being invalidate because project 8b3d9dfa1c2f4b75923b5e6fbc4498e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:19:35.946127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-86e41715-7597-49af-91e9-dc7a9054603e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:35.947026 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-86e41715-7597-49af-91e9-dc7a9054603e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:35.949239 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1266/2527] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:19:35 2022] DELETE /identity/v3/projects/8b3d9dfa1c2f4b75923b5e6fbc4498e3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:35.954042 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07355f-cbb2-4c78-831a-6fe97a74eb3f service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:35.966667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07355f-cbb2-4c78-831a-6fe97a74eb3f service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:36.089182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2337d-c552-458b-9676-389e81d57dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:36.090216 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2337d-c552-458b-9676-389e81d57dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:36.090585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2337d-c552-458b-9676-389e81d57dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:36.090905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2337d-c552-458b-9676-389e81d57dab admin admin] PATH_INFO: `/v3/users/c07bff8c2c32480583989f2a7606395d` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:36.092790 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2337d-c552-458b-9676-389e81d57dab admin admin] RBAC: Authorizing `identity:delete_user(user_id=c07bff8c2c32480583989f2a7606395d)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:36.095201 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2337d-c552-458b-9676-389e81d57dab admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:36.151038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-deb2337d-c552-458b-9676-389e81d57dab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c07bff8c2c32480583989f2a7606395d'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:36.167505 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-deb2337d-c552-458b-9676-389e81d57dab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c07bff8c2c32480583989f2a7606395d'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:36.175445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-deb2337d-c552-458b-9676-389e81d57dab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c07bff8c2c32480583989f2a7606395d'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:36.176204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-deb2337d-c552-458b-9676-389e81d57dab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c07bff8c2c32480583989f2a7606395d'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:36.183335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-deb2337d-c552-458b-9676-389e81d57dab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c07bff8c2c32480583989f2a7606395d'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:36.191542 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1262/2528] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:19:35 2022] DELETE /identity/v3/users/c07bff8c2c32480583989f2a7606395d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:36.469618 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e41715-7597-49af-91e9-dc7a9054603e admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:36.471253 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-86e41715-7597-49af-91e9-dc7a9054603e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:36.488894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e41715-7597-49af-91e9-dc7a9054603e admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:36.497495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:36.557586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af7897-55c8-4ca9-9c4c-4083828e4721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:36.558580 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af7897-55c8-4ca9-9c4c-4083828e4721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:36.558907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af7897-55c8-4ca9-9c4c-4083828e4721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:36.559190 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2af7897-55c8-4ca9-9c4c-4083828e4721 admin admin] PATH_INFO: `/v3/projects/28584fe80f8340cf8958b66d3aa625fa` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:36.560382 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af7897-55c8-4ca9-9c4c-4083828e4721 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28584fe80f8340cf8958b66d3aa625fa)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:36.563127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2af7897-55c8-4ca9-9c4c-4083828e4721 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:36.631262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d2af7897-55c8-4ca9-9c4c-4083828e4721 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28584fe80f8340cf8958b66d3aa625fa', 'request_id': 'req-d2af7897-55c8-4ca9-9c4c-4083828e4721'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:36.644797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d2af7897-55c8-4ca9-9c4c-4083828e4721 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28584fe80f8340cf8958b66d3aa625fa', 'request_id': 'req-d2af7897-55c8-4ca9-9c4c-4083828e4721'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:36.651460 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d2af7897-55c8-4ca9-9c4c-4083828e4721 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28584fe80f8340cf8958b66d3aa625fa', 'request_id': 'req-d2af7897-55c8-4ca9-9c4c-4083828e4721'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:36.664056 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d2af7897-55c8-4ca9-9c4c-4083828e4721 admin admin] The token cache is being invalidate because project 28584fe80f8340cf8958b66d3aa625fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:19:36.664347 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d2af7897-55c8-4ca9-9c4c-4083828e4721 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:36.667977 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d2af7897-55c8-4ca9-9c4c-4083828e4721 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:36.669592 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1267/2529] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:19:36 2022] DELETE /identity/v3/projects/28584fe80f8340cf8958b66d3aa625fa => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:36.677047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2337d-c552-458b-9676-389e81d57dab admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:36.679520 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-deb2337d-c552-458b-9676-389e81d57dab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:36.693401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2337d-c552-458b-9676-389e81d57dab admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:36.763260 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:36.775290 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec3271-979a-4e68-af9d-705ee4a4ecca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:36.776419 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec3271-979a-4e68-af9d-705ee4a4ecca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:36.776749 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec3271-979a-4e68-af9d-705ee4a4ecca admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:36.777025 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec3271-979a-4e68-af9d-705ee4a4ecca admin admin] PATH_INFO: `/v3/users/7d25e8b831fc436da755fdff357e9d1c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:36.778223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec3271-979a-4e68-af9d-705ee4a4ecca admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d25e8b831fc436da755fdff357e9d1c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:36.781227 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec3271-979a-4e68-af9d-705ee4a4ecca admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:36.834804 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-64ec3271-979a-4e68-af9d-705ee4a4ecca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d25e8b831fc436da755fdff357e9d1c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:36.850845 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-64ec3271-979a-4e68-af9d-705ee4a4ecca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d25e8b831fc436da755fdff357e9d1c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:36.860119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-64ec3271-979a-4e68-af9d-705ee4a4ecca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d25e8b831fc436da755fdff357e9d1c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:36.860923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-64ec3271-979a-4e68-af9d-705ee4a4ecca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d25e8b831fc436da755fdff357e9d1c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:36.868859 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-64ec3271-979a-4e68-af9d-705ee4a4ecca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d25e8b831fc436da755fdff357e9d1c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:36.879848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1263/2530] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:19:36 2022] DELETE /identity/v3/users/7d25e8b831fc436da755fdff357e9d1c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:37.168790 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af7897-55c8-4ca9-9c4c-4083828e4721 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:37.170385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d2af7897-55c8-4ca9-9c4c-4083828e4721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:37.193903 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2af7897-55c8-4ca9-9c4c-4083828e4721 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:37.266234 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2b61d-84b4-453a-855b-93e6a6d09d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:37.267188 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2b61d-84b4-453a-855b-93e6a6d09d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:37.267476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2b61d-84b4-453a-855b-93e6a6d09d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:37.267757 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2b61d-84b4-453a-855b-93e6a6d09d0e admin admin] PATH_INFO: `/v3/projects/4e8a00b1f2db4cd39f005239c362f9a6` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:37.269003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2b61d-84b4-453a-855b-93e6a6d09d0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e8a00b1f2db4cd39f005239c362f9a6)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:37.271006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2b61d-84b4-453a-855b-93e6a6d09d0e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:37.330381 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:37.336348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-61b2b61d-84b4-453a-855b-93e6a6d09d0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e8a00b1f2db4cd39f005239c362f9a6', 'request_id': 'req-61b2b61d-84b4-453a-855b-93e6a6d09d0e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:37.350937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-61b2b61d-84b4-453a-855b-93e6a6d09d0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e8a00b1f2db4cd39f005239c362f9a6', 'request_id': 'req-61b2b61d-84b4-453a-855b-93e6a6d09d0e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:37.358399 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-61b2b61d-84b4-453a-855b-93e6a6d09d0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e8a00b1f2db4cd39f005239c362f9a6', 'request_id': 'req-61b2b61d-84b4-453a-855b-93e6a6d09d0e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:37.377822 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-61b2b61d-84b4-453a-855b-93e6a6d09d0e admin admin] The token cache is being invalidate because project 4e8a00b1f2db4cd39f005239c362f9a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:19:37.378157 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-61b2b61d-84b4-453a-855b-93e6a6d09d0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:37.378953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-61b2b61d-84b4-453a-855b-93e6a6d09d0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:37.380903 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1268/2531] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:19:37 2022] DELETE /identity/v3/projects/4e8a00b1f2db4cd39f005239c362f9a6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:37.386921 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bc0f7-8cff-4ad7-9431-9267ec4ee9e0 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:37.387772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bc0f7-8cff-4ad7-9431-9267ec4ee9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:37.387772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bc0f7-8cff-4ad7-9431-9267ec4ee9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:37.408680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-609bc0f7-8cff-4ad7-9431-9267ec4ee9e0 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:37.415107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-609bc0f7-8cff-4ad7-9431-9267ec4ee9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:37.483956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1264/2532] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:37.492335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdaac5-3258-43bc-913e-f904bf40bf53 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:37.492842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdaac5-3258-43bc-913e-f904bf40bf53 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:37.492842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdaac5-3258-43bc-913e-f904bf40bf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:37.511671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-ccfdaac5-3258-43bc-913e-f904bf40bf53 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:37.512958 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ccfdaac5-3258-43bc-913e-f904bf40bf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:37.523700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1269/2533] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:37.528162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609bc0f7-8cff-4ad7-9431-9267ec4ee9e0 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:37.538013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609bc0f7-8cff-4ad7-9431-9267ec4ee9e0 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:37.542604 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9e246-211a-446c-98d6-717033a217ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:37.543290 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9e246-211a-446c-98d6-717033a217ce admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:37.543578 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9e246-211a-446c-98d6-717033a217ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:37.543856 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9e246-211a-446c-98d6-717033a217ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:37.544584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9e246-211a-446c-98d6-717033a217ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:19:37.544893 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9e246-211a-446c-98d6-717033a217ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:37.547073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9e246-211a-446c-98d6-717033a217ce admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:37.562928 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1265/2534] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:19:37.567337 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfdaac5-3258-43bc-913e-f904bf40bf53 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:37.577142 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfdaac5-3258-43bc-913e-f904bf40bf53 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:37.582043 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7360e7-c37d-4dae-8989-6cc429e63353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:37.582728 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7360e7-c37d-4dae-8989-6cc429e63353 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:37.583156 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7360e7-c37d-4dae-8989-6cc429e63353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:37.583444 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7360e7-c37d-4dae-8989-6cc429e63353 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:37.584152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7360e7-c37d-4dae-8989-6cc429e63353 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:37.586129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7360e7-c37d-4dae-8989-6cc429e63353 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:37.612410 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1270/2535] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:19:37 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:19:37.615995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9e246-211a-446c-98d6-717033a217ce admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:37.626987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9e246-211a-446c-98d6-717033a217ce admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:37.688763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e68b6b-ef6a-436b-97dc-11ff0fa19668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:37.689444 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e68b6b-ef6a-436b-97dc-11ff0fa19668 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:37.689732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e68b6b-ef6a-436b-97dc-11ff0fa19668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:37.690011 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e68b6b-ef6a-436b-97dc-11ff0fa19668 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:37.690739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e68b6b-ef6a-436b-97dc-11ff0fa19668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:37.692692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e68b6b-ef6a-436b-97dc-11ff0fa19668 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:37.700222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:37.718962 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1266/2536] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:19:37 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:19:37.720344 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7360e7-c37d-4dae-8989-6cc429e63353 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:37.731120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7360e7-c37d-4dae-8989-6cc429e63353 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:37.736558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fffdfc-c6a1-4e8d-8aed-098ad0fae529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:37.737366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fffdfc-c6a1-4e8d-8aed-098ad0fae529 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:37.737525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fffdfc-c6a1-4e8d-8aed-098ad0fae529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:37.737810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fffdfc-c6a1-4e8d-8aed-098ad0fae529 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:37.738551 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fffdfc-c6a1-4e8d-8aed-098ad0fae529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:37.740175 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fffdfc-c6a1-4e8d-8aed-098ad0fae529 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:37.751211 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1271/2537] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:37.755696 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e68b6b-ef6a-436b-97dc-11ff0fa19668 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:37.768061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e68b6b-ef6a-436b-97dc-11ff0fa19668 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:37.772991 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57835b-6922-4609-9970-a1afca77ce11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:37.773671 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57835b-6922-4609-9970-a1afca77ce11 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:37.773960 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57835b-6922-4609-9970-a1afca77ce11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:37.774245 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57835b-6922-4609-9970-a1afca77ce11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:37.775674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57835b-6922-4609-9970-a1afca77ce11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:37.777290 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57835b-6922-4609-9970-a1afca77ce11 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:37.789211 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1267/2538] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:37.793232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fffdfc-c6a1-4e8d-8aed-098ad0fae529 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:37.804537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fffdfc-c6a1-4e8d-8aed-098ad0fae529 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:37.809094 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a78d9-de4a-43a9-b817-f09f90bc611c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:37.809974 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a78d9-de4a-43a9-b817-f09f90bc611c admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:37.810257 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a78d9-de4a-43a9-b817-f09f90bc611c admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:37.810531 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a78d9-de4a-43a9-b817-f09f90bc611c admin admin] PATH_INFO: `/v3/projects/44bf06927ad44c4b88d54f786781be68/users/7c367c44c73d4f2790e37ebaeb5c7192/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:37.826935 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a78d9-de4a-43a9-b817-f09f90bc611c admin admin] RBAC: Authorizing `identity:create_grant(project_id=44bf06927ad44c4b88d54f786781be68, user_id=7c367c44c73d4f2790e37ebaeb5c7192, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:37.829675 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a78d9-de4a-43a9-b817-f09f90bc611c admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:37.842905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1272/2539] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:37 2022] PUT /identity/v3/projects/44bf06927ad44c4b88d54f786781be68/users/7c367c44c73d4f2790e37ebaeb5c7192/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:37.848987 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3080842-64a7-468d-9ae5-9a2c85c096d2 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:37.849286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3080842-64a7-468d-9ae5-9a2c85c096d2 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:37.849573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3080842-64a7-468d-9ae5-9a2c85c096d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:37.867993 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-f3080842-64a7-468d-9ae5-9a2c85c096d2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:37.869198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f3080842-64a7-468d-9ae5-9a2c85c096d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:37.928689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1268/2540] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:37.938566 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a78d9-de4a-43a9-b817-f09f90bc611c admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:37.940305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-cd1a78d9-de4a-43a9-b817-f09f90bc611c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:37.998980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a78d9-de4a-43a9-b817-f09f90bc611c admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:38.002900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d101e2b-2dd2-4846-bb05-6ade5fad3618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:38.003501 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101e2b-2dd2-4846-bb05-6ade5fad3618 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:38.003789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101e2b-2dd2-4846-bb05-6ade5fad3618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:38.004069 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101e2b-2dd2-4846-bb05-6ade5fad3618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:38.013443 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d101e2b-2dd2-4846-bb05-6ade5fad3618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:38.015423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d101e2b-2dd2-4846-bb05-6ade5fad3618 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:38.028727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1273/2541] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:38.775938 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d38149-ed3f-4fd1-af81-94e87cc23610 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:38.776478 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d38149-ed3f-4fd1-af81-94e87cc23610 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:38.776690 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d38149-ed3f-4fd1-af81-94e87cc23610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:38.809107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-80d38149-ed3f-4fd1-af81-94e87cc23610 None None] MFA Rules not processed for user `7c367c44c73d4f2790e37ebaeb5c7192`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:38.810335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-80d38149-ed3f-4fd1-af81-94e87cc23610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:38.893342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1269/2542] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:38.897902 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d101e2b-2dd2-4846-bb05-6ade5fad3618 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:38.908710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d101e2b-2dd2-4846-bb05-6ade5fad3618 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:38.913326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fac57a-52ce-42e8-a38f-99a31816da9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:38.913994 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fac57a-52ce-42e8-a38f-99a31816da9a admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:38.914285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fac57a-52ce-42e8-a38f-99a31816da9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:38.914570 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fac57a-52ce-42e8-a38f-99a31816da9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:38.915346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fac57a-52ce-42e8-a38f-99a31816da9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:38.917259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fac57a-52ce-42e8-a38f-99a31816da9a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:38.960392 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d38149-ed3f-4fd1-af81-94e87cc23610 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:38.962759 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1274/2543] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:19:38 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:38.973556 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d38149-ed3f-4fd1-af81-94e87cc23610 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:39.040867 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ce131-8e83-4225-9646-435891efee60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:39.041568 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ce131-8e83-4225-9646-435891efee60 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:39.041847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ce131-8e83-4225-9646-435891efee60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:39.042126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ce131-8e83-4225-9646-435891efee60 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:39.042988 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ce131-8e83-4225-9646-435891efee60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:39.046868 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ce131-8e83-4225-9646-435891efee60 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:39.059920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:39.071867 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fac57a-52ce-42e8-a38f-99a31816da9a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:39.075025 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1270/2544] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:19:38 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:39.087289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fac57a-52ce-42e8-a38f-99a31816da9a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:39.092835 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017600a2-a746-40b7-a6ec-25189c27d727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:39.093520 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017600a2-a746-40b7-a6ec-25189c27d727 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:39.093837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017600a2-a746-40b7-a6ec-25189c27d727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:39.094123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017600a2-a746-40b7-a6ec-25189c27d727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:39.094966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017600a2-a746-40b7-a6ec-25189c27d727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:39.096710 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017600a2-a746-40b7-a6ec-25189c27d727 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:39.108457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1275/2545] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:19:39.115395 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ce131-8e83-4225-9646-435891efee60 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:39.126182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ce131-8e83-4225-9646-435891efee60 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:39.130919 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df3328-c202-4ee1-97f6-8826009f8338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:39.131870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df3328-c202-4ee1-97f6-8826009f8338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:39.132217 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df3328-c202-4ee1-97f6-8826009f8338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:39.132559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df3328-c202-4ee1-97f6-8826009f8338 admin admin] PATH_INFO: `/v3/projects/0c7042d786744a53a1b9f0ef7009dc46/users/4ccfc1a5399c44459213698977990536/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:39.151591 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df3328-c202-4ee1-97f6-8826009f8338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c7042d786744a53a1b9f0ef7009dc46, user_id=4ccfc1a5399c44459213698977990536, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:39.154339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df3328-c202-4ee1-97f6-8826009f8338 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:39.169882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1271/2546] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:39 2022] PUT /identity/v3/projects/0c7042d786744a53a1b9f0ef7009dc46/users/4ccfc1a5399c44459213698977990536/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:39.174556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017600a2-a746-40b7-a6ec-25189c27d727 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:39.185029 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017600a2-a746-40b7-a6ec-25189c27d727 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:39.231006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:39.262304 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa15d2f-76b0-48bb-988a-2af66550e59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:39.263013 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa15d2f-76b0-48bb-988a-2af66550e59e admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:39.263321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa15d2f-76b0-48bb-988a-2af66550e59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:39.263601 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa15d2f-76b0-48bb-988a-2af66550e59e admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:39.264436 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa15d2f-76b0-48bb-988a-2af66550e59e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:39.266074 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa15d2f-76b0-48bb-988a-2af66550e59e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:39.280199 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1276/2547] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:39.281988 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df3328-c202-4ee1-97f6-8826009f8338 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:39.292945 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df3328-c202-4ee1-97f6-8826009f8338 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:39.300123 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d27861-dd3f-4149-9fe2-f5c82d7bf3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:39.301214 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d27861-dd3f-4149-9fe2-f5c82d7bf3ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:39.301499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d27861-dd3f-4149-9fe2-f5c82d7bf3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:39.301782 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d27861-dd3f-4149-9fe2-f5c82d7bf3ff admin admin] PATH_INFO: `/v3/projects/0c7042d786744a53a1b9f0ef7009dc46/users/4ccfc1a5399c44459213698977990536/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:39.311387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d27861-dd3f-4149-9fe2-f5c82d7bf3ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c7042d786744a53a1b9f0ef7009dc46, user_id=4ccfc1a5399c44459213698977990536, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:39.313691 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d27861-dd3f-4149-9fe2-f5c82d7bf3ff admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:39.326719 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1272/2548] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:39 2022] PUT /identity/v3/projects/0c7042d786744a53a1b9f0ef7009dc46/users/4ccfc1a5399c44459213698977990536/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:40.226745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504d431-c11f-418f-b88e-9076ee26beec None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:40.226745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504d431-c11f-418f-b88e-9076ee26beec None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:40.227416 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504d431-c11f-418f-b88e-9076ee26beec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:40.258224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-7504d431-c11f-418f-b88e-9076ee26beec None None] MFA Rules not processed for user `4ccfc1a5399c44459213698977990536`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:40.260053 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7504d431-c11f-418f-b88e-9076ee26beec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:40.355607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1277/2549] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:40 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:40.369461 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d27861-dd3f-4149-9fe2-f5c82d7bf3ff admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:40.372013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-47d27861-dd3f-4149-9fe2-f5c82d7bf3ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:40.531879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d27861-dd3f-4149-9fe2-f5c82d7bf3ff admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:40.550979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce164a-e5f5-4605-84d9-bf497961940b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:40.551945 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce164a-e5f5-4605-84d9-bf497961940b service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:40.552235 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce164a-e5f5-4605-84d9-bf497961940b service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:40.552519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce164a-e5f5-4605-84d9-bf497961940b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:40.665837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce164a-e5f5-4605-84d9-bf497961940b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:40.672292 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce164a-e5f5-4605-84d9-bf497961940b service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:40.688462 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1273/2550] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:42.389248 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504d431-c11f-418f-b88e-9076ee26beec None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:42.411576 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504d431-c11f-418f-b88e-9076ee26beec None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:42.417270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69bdfe-ab53-4bf6-a671-58546bec331c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:42.418088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69bdfe-ab53-4bf6-a671-58546bec331c service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:42.418480 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69bdfe-ab53-4bf6-a671-58546bec331c service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:42.418987 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69bdfe-ab53-4bf6-a671-58546bec331c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:42.442859 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69bdfe-ab53-4bf6-a671-58546bec331c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:42.447209 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e69bdfe-ab53-4bf6-a671-58546bec331c service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:42.468124 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1278/2551] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:42 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:44.900787 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce164a-e5f5-4605-84d9-bf497961940b service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:44.902213 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-99ce164a-e5f5-4605-84d9-bf497961940b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:44.959665 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce164a-e5f5-4605-84d9-bf497961940b service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:44.963261 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed910e-9579-4a5e-8bf3-67e7013804e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:44.963907 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed910e-9579-4a5e-8bf3-67e7013804e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:44.964193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed910e-9579-4a5e-8bf3-67e7013804e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:44.964482 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed910e-9579-4a5e-8bf3-67e7013804e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:44.966250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-eaed910e-9579-4a5e-8bf3-67e7013804e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:45.115990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:45.151631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed910e-9579-4a5e-8bf3-67e7013804e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:45.153057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed910e-9579-4a5e-8bf3-67e7013804e1 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:45.163080 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1274/2552] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:46.865567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69bdfe-ab53-4bf6-a671-58546bec331c service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:46.875907 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e69bdfe-ab53-4bf6-a671-58546bec331c service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:46.879715 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbe08a-7e03-4c1f-936e-e1193719e470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:46.880366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbe08a-7e03-4c1f-936e-e1193719e470 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:46.880650 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbe08a-7e03-4c1f-936e-e1193719e470 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:46.880925 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbe08a-7e03-4c1f-936e-e1193719e470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:46.882744 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-71fbe08a-7e03-4c1f-936e-e1193719e470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:46.895449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbe08a-7e03-4c1f-936e-e1193719e470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:46.896894 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fbe08a-7e03-4c1f-936e-e1193719e470 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:46.907624 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1279/2553] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:48.350600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed910e-9579-4a5e-8bf3-67e7013804e1 service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:48.353632 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-eaed910e-9579-4a5e-8bf3-67e7013804e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:48.439995 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbe08a-7e03-4c1f-936e-e1193719e470 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:48.441499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-71fbe08a-7e03-4c1f-936e-e1193719e470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:48.465195 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbe08a-7e03-4c1f-936e-e1193719e470 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:48.487162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e5a33-58b3-402f-932e-7ab29f8ec55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:48.489344 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e5a33-58b3-402f-932e-7ab29f8ec55a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:48.489869 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e5a33-58b3-402f-932e-7ab29f8ec55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:48.490433 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e5a33-58b3-402f-932e-7ab29f8ec55a admin admin] PATH_INFO: `/v3/users/f0547075e10c498b8d13e1859197cea0` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:48.492677 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e5a33-58b3-402f-932e-7ab29f8ec55a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0547075e10c498b8d13e1859197cea0)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:48.495316 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e5a33-58b3-402f-932e-7ab29f8ec55a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:48.516828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed910e-9579-4a5e-8bf3-67e7013804e1 service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:48.524470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8238e63-80f8-48d9-9ddd-b39a9a2f41ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '066f69a87f5f48c7b20c1e0fcf5c1107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:48.525166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8238e63-80f8-48d9-9ddd-b39a9a2f41ca service placement] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:48.525448 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8238e63-80f8-48d9-9ddd-b39a9a2f41ca service placement] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:48.526471 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8238e63-80f8-48d9-9ddd-b39a9a2f41ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:48.533838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d8238e63-80f8-48d9-9ddd-b39a9a2f41ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:48.542473 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8238e63-80f8-48d9-9ddd-b39a9a2f41ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:48.544153 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8238e63-80f8-48d9-9ddd-b39a9a2f41ca service placement] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:48.559337 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1275/2554] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:48.570719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-854e5a33-58b3-402f-932e-7ab29f8ec55a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0547075e10c498b8d13e1859197cea0'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:48.591218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-854e5a33-58b3-402f-932e-7ab29f8ec55a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0547075e10c498b8d13e1859197cea0'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:48.600373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-854e5a33-58b3-402f-932e-7ab29f8ec55a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0547075e10c498b8d13e1859197cea0'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:48.601113 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-854e5a33-58b3-402f-932e-7ab29f8ec55a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0547075e10c498b8d13e1859197cea0'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:48.610869 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-854e5a33-58b3-402f-932e-7ab29f8ec55a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0547075e10c498b8d13e1859197cea0'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:48.617565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1280/2555] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:19:48 2022] DELETE /identity/v3/users/f0547075e10c498b8d13e1859197cea0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:19:48.835876 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8238e63-80f8-48d9-9ddd-b39a9a2f41ca service placement] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:48.855530 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8238e63-80f8-48d9-9ddd-b39a9a2f41ca service placement] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:48.922283 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95dc638-025f-463d-ae4b-21dd95e53b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:48.923482 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dc638-025f-463d-ae4b-21dd95e53b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:48.923768 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dc638-025f-463d-ae4b-21dd95e53b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:48.924047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dc638-025f-463d-ae4b-21dd95e53b40 admin admin] PATH_INFO: `/v3/projects/752497421a0b4c70a08bcca4393b8837` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:48.925275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95dc638-025f-463d-ae4b-21dd95e53b40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=752497421a0b4c70a08bcca4393b8837)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:48.927408 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95dc638-025f-463d-ae4b-21dd95e53b40 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:48.987717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d95dc638-025f-463d-ae4b-21dd95e53b40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '752497421a0b4c70a08bcca4393b8837', 'request_id': 'req-d95dc638-025f-463d-ae4b-21dd95e53b40'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:48.994890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:48.996569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d95dc638-025f-463d-ae4b-21dd95e53b40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '752497421a0b4c70a08bcca4393b8837', 'request_id': 'req-d95dc638-025f-463d-ae4b-21dd95e53b40'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:49.010424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d95dc638-025f-463d-ae4b-21dd95e53b40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '752497421a0b4c70a08bcca4393b8837', 'request_id': 'req-d95dc638-025f-463d-ae4b-21dd95e53b40'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:49.023576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d95dc638-025f-463d-ae4b-21dd95e53b40 admin admin] The token cache is being invalidate because project 752497421a0b4c70a08bcca4393b8837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:19:49.023770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d95dc638-025f-463d-ae4b-21dd95e53b40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:49.024455 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d95dc638-025f-463d-ae4b-21dd95e53b40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:49.026000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1276/2556] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:19:48 2022] DELETE /identity/v3/projects/752497421a0b4c70a08bcca4393b8837 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:49.031873 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f98f34-c7f1-46c6-adb3-7965a0d4e36f None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:49.032457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f98f34-c7f1-46c6-adb3-7965a0d4e36f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:49.032738 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f98f34-c7f1-46c6-adb3-7965a0d4e36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:49.041486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:49.056869 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-46f98f34-c7f1-46c6-adb3-7965a0d4e36f None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:49.058890 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-46f98f34-c7f1-46c6-adb3-7965a0d4e36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:49.123817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1281/2557] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:19:49.131717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ece66-ac8a-4393-bad8-63206f8e84af None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:49.132478 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ece66-ac8a-4393-bad8-63206f8e84af None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:49.132478 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ece66-ac8a-4393-bad8-63206f8e84af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:49.153956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-431ece66-ac8a-4393-bad8-63206f8e84af None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:49.155531 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-431ece66-ac8a-4393-bad8-63206f8e84af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:49.166414 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1277/2558] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:49.170391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f98f34-c7f1-46c6-adb3-7965a0d4e36f None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:49.180993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f98f34-c7f1-46c6-adb3-7965a0d4e36f None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:49.187585 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66835a81-1642-41af-80f6-babee2987829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:49.188435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66835a81-1642-41af-80f6-babee2987829 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:49.188819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66835a81-1642-41af-80f6-babee2987829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:49.189097 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66835a81-1642-41af-80f6-babee2987829 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:49.189862 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66835a81-1642-41af-80f6-babee2987829 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:19:49.190381 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66835a81-1642-41af-80f6-babee2987829 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:49.192413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66835a81-1642-41af-80f6-babee2987829 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:49.208344 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1282/2559] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:19:49.213226 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ece66-ac8a-4393-bad8-63206f8e84af None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:49.223968 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ece66-ac8a-4393-bad8-63206f8e84af None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:49.230417 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1f087-d03c-4ef8-a368-3617d73ffe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:49.231332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1f087-d03c-4ef8-a368-3617d73ffe5a admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:49.231618 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1f087-d03c-4ef8-a368-3617d73ffe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:49.231895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1f087-d03c-4ef8-a368-3617d73ffe5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:49.232600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1f087-d03c-4ef8-a368-3617d73ffe5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:49.234890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1f087-d03c-4ef8-a368-3617d73ffe5a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:49.265010 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1278/2560] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:19:49 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:49.270421 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66835a81-1642-41af-80f6-babee2987829 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:49.284301 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66835a81-1642-41af-80f6-babee2987829 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:49.345861 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fb9de-d62c-4489-a510-9f206d752d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:49.346841 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fb9de-d62c-4489-a510-9f206d752d32 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:49.347134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fb9de-d62c-4489-a510-9f206d752d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:49.347430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fb9de-d62c-4489-a510-9f206d752d32 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:49.348298 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94fb9de-d62c-4489-a510-9f206d752d32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:49.350688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94fb9de-d62c-4489-a510-9f206d752d32 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:49.384477 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1283/2561] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:19:49 2022] POST /identity/v3/users => generated 283 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:49.385612 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1f087-d03c-4ef8-a368-3617d73ffe5a admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:49.397935 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1f087-d03c-4ef8-a368-3617d73ffe5a admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:49.402754 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f43112-12c1-420d-ab81-f2b6da346fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:49.404989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f43112-12c1-420d-ab81-f2b6da346fda admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:49.405255 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f43112-12c1-420d-ab81-f2b6da346fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:49.405531 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f43112-12c1-420d-ab81-f2b6da346fda admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:49.406663 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f43112-12c1-420d-ab81-f2b6da346fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:49.408311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f43112-12c1-420d-ab81-f2b6da346fda admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:49.421109 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1279/2562] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:49.425493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fb9de-d62c-4489-a510-9f206d752d32 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:49.436722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fb9de-d62c-4489-a510-9f206d752d32 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:49.442661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc09ead-8f66-48b2-83fb-d43a70d58d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:49.444364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc09ead-8f66-48b2-83fb-d43a70d58d49 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:49.444831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc09ead-8f66-48b2-83fb-d43a70d58d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:49.445261 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc09ead-8f66-48b2-83fb-d43a70d58d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:49.446166 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc09ead-8f66-48b2-83fb-d43a70d58d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:49.447958 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc09ead-8f66-48b2-83fb-d43a70d58d49 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:49.472997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1284/2563] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:19:49.477389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f43112-12c1-420d-ab81-f2b6da346fda admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:49.493548 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f43112-12c1-420d-ab81-f2b6da346fda admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:49.501960 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1b132-662d-4cf7-a8ab-6f8d1c55f42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:49.503125 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1b132-662d-4cf7-a8ab-6f8d1c55f42f admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:49.503620 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1b132-662d-4cf7-a8ab-6f8d1c55f42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:49.504199 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1b132-662d-4cf7-a8ab-6f8d1c55f42f admin admin] PATH_INFO: `/v3/projects/3a8659d2936746b2a89772c02e01e248/users/7220994b9eed4aa680dfa2ee1496d70f/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:49.530898 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1b132-662d-4cf7-a8ab-6f8d1c55f42f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a8659d2936746b2a89772c02e01e248, user_id=7220994b9eed4aa680dfa2ee1496d70f, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:49.538773 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1b132-662d-4cf7-a8ab-6f8d1c55f42f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:49.564664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1280/2564] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:49 2022] PUT /identity/v3/projects/3a8659d2936746b2a89772c02e01e248/users/7220994b9eed4aa680dfa2ee1496d70f/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:49.574487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e552b4-a20d-4056-90a0-a792bc71bfe1 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:49.574794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e552b4-a20d-4056-90a0-a792bc71bfe1 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:49.575465 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e552b4-a20d-4056-90a0-a792bc71bfe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:49.615548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-00e552b4-a20d-4056-90a0-a792bc71bfe1 None None] MFA Rules not processed for user `7220994b9eed4aa680dfa2ee1496d70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:49.617456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-00e552b4-a20d-4056-90a0-a792bc71bfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:49.742035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:49.762697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1285/2565] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:49 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:49.763504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1b132-662d-4cf7-a8ab-6f8d1c55f42f admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:49.893728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1b132-662d-4cf7-a8ab-6f8d1c55f42f admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:49.903861 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d0044-792e-4417-8abe-8dfc4ffdec36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:49.904680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d0044-792e-4417-8abe-8dfc4ffdec36 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:49.905085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d0044-792e-4417-8abe-8dfc4ffdec36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:49.905563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d0044-792e-4417-8abe-8dfc4ffdec36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:49.906408 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d0044-792e-4417-8abe-8dfc4ffdec36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:49.908510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d0044-792e-4417-8abe-8dfc4ffdec36 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:49.956209 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1281/2566] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:19:49 2022] POST /identity/v3/projects => generated 363 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:19:49.959976 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e552b4-a20d-4056-90a0-a792bc71bfe1 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:49.983706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e552b4-a20d-4056-90a0-a792bc71bfe1 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:50.097762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c72d37-d821-421b-afab-af9977519cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:50.098756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c72d37-d821-421b-afab-af9977519cee admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:50.099774 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c72d37-d821-421b-afab-af9977519cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:50.100311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c72d37-d821-421b-afab-af9977519cee admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:50.101848 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c72d37-d821-421b-afab-af9977519cee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:50.104706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c72d37-d821-421b-afab-af9977519cee admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:50.150864 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1286/2567] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:19:49 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:50.154805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d0044-792e-4417-8abe-8dfc4ffdec36 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:50.173225 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d0044-792e-4417-8abe-8dfc4ffdec36 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:50.177516 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fd42d-f317-48b4-ad1c-5bc0bd8b9b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:50.178282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fd42d-f317-48b4-ad1c-5bc0bd8b9b0a admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:50.178633 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fd42d-f317-48b4-ad1c-5bc0bd8b9b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:50.179062 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fd42d-f317-48b4-ad1c-5bc0bd8b9b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:50.179922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43fd42d-f317-48b4-ad1c-5bc0bd8b9b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:50.181596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43fd42d-f317-48b4-ad1c-5bc0bd8b9b0a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:50.197265 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1282/2568] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:19:50.200494 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c72d37-d821-421b-afab-af9977519cee admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:50.214903 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c72d37-d821-421b-afab-af9977519cee admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:50.226392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66757b3e-6857-4348-91b0-a18e337fe3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:50.227452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66757b3e-6857-4348-91b0-a18e337fe3d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:50.227821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66757b3e-6857-4348-91b0-a18e337fe3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:50.228170 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66757b3e-6857-4348-91b0-a18e337fe3d1 admin admin] PATH_INFO: `/v3/projects/908e393471644625b06e99ae26d95ebf/users/eb5e37a89dfd4911a0af70f7feb95fbb/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:50.276426 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66757b3e-6857-4348-91b0-a18e337fe3d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=908e393471644625b06e99ae26d95ebf, user_id=eb5e37a89dfd4911a0af70f7feb95fbb, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:50.280348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66757b3e-6857-4348-91b0-a18e337fe3d1 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:50.309144 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1287/2569] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:50 2022] PUT /identity/v3/projects/908e393471644625b06e99ae26d95ebf/users/eb5e37a89dfd4911a0af70f7feb95fbb/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:50.313686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fd42d-f317-48b4-ad1c-5bc0bd8b9b0a admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:50.325558 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fd42d-f317-48b4-ad1c-5bc0bd8b9b0a admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:50.402346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:50.415591 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70718b68-7322-4b32-9010-e36f950ced84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:50.418108 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70718b68-7322-4b32-9010-e36f950ced84 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:50.418824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70718b68-7322-4b32-9010-e36f950ced84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:50.419319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70718b68-7322-4b32-9010-e36f950ced84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:50.420308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70718b68-7322-4b32-9010-e36f950ced84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:50.424094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70718b68-7322-4b32-9010-e36f950ced84 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:50.455908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1283/2570] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:50 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:50.459943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66757b3e-6857-4348-91b0-a18e337fe3d1 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:50.478063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66757b3e-6857-4348-91b0-a18e337fe3d1 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:50.492332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47dc7bd-d594-4818-8e96-2c6c9e2ad8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:50.493259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dc7bd-d594-4818-8e96-2c6c9e2ad8e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:50.493533 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dc7bd-d594-4818-8e96-2c6c9e2ad8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:50.493833 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dc7bd-d594-4818-8e96-2c6c9e2ad8e9 admin admin] PATH_INFO: `/v3/projects/908e393471644625b06e99ae26d95ebf/users/eb5e37a89dfd4911a0af70f7feb95fbb/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:50.504277 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47dc7bd-d594-4818-8e96-2c6c9e2ad8e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=908e393471644625b06e99ae26d95ebf, user_id=eb5e37a89dfd4911a0af70f7feb95fbb, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:50.508445 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47dc7bd-d594-4818-8e96-2c6c9e2ad8e9 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:50.537660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1288/2571] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:50 2022] PUT /identity/v3/projects/908e393471644625b06e99ae26d95ebf/users/eb5e37a89dfd4911a0af70f7feb95fbb/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:50.552036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d966fc-a950-413c-9714-994b4debee61 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:50.552327 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d966fc-a950-413c-9714-994b4debee61 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:50.552607 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d966fc-a950-413c-9714-994b4debee61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:50.586585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-06d966fc-a950-413c-9714-994b4debee61 None None] MFA Rules not processed for user `eb5e37a89dfd4911a0af70f7feb95fbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:50.589567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-06d966fc-a950-413c-9714-994b4debee61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:50.709948 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1284/2572] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:50 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:50.725319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47dc7bd-d594-4818-8e96-2c6c9e2ad8e9 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:50.727715 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e47dc7bd-d594-4818-8e96-2c6c9e2ad8e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:50.772126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d966fc-a950-413c-9714-994b4debee61 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:50.773932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-06d966fc-a950-413c-9714-994b4debee61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:50.821648 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47dc7bd-d594-4818-8e96-2c6c9e2ad8e9 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:50.826944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fb28e-7ec4-4643-b810-c86f928b99ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:50.827705 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fb28e-7ec4-4643-b810-c86f928b99ff service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:50.828955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fb28e-7ec4-4643-b810-c86f928b99ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:50.829185 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74fb28e-7ec4-4643-b810-c86f928b99ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:50.839319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fb28e-7ec4-4643-b810-c86f928b99ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:50.840764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74fb28e-7ec4-4643-b810-c86f928b99ff service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:50.853006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1289/2573] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:50 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:50.872308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d966fc-a950-413c-9714-994b4debee61 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:50.880303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd134f5-c817-4f6f-b861-b0d9562b90ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:50.884081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd134f5-c817-4f6f-b861-b0d9562b90ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:50.884502 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd134f5-c817-4f6f-b861-b0d9562b90ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:50.884795 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd134f5-c817-4f6f-b861-b0d9562b90ff admin admin] PATH_INFO: `/v3/users/0a3ed8bea97e49628311cb45d48ca699` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:50.891060 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd134f5-c817-4f6f-b861-b0d9562b90ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a3ed8bea97e49628311cb45d48ca699)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:50.893562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd134f5-c817-4f6f-b861-b0d9562b90ff admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:50.959052 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-7fd134f5-c817-4f6f-b861-b0d9562b90ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a3ed8bea97e49628311cb45d48ca699'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:50.974328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-7fd134f5-c817-4f6f-b861-b0d9562b90ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a3ed8bea97e49628311cb45d48ca699'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:50.983388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-7fd134f5-c817-4f6f-b861-b0d9562b90ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a3ed8bea97e49628311cb45d48ca699'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:50.984412 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-7fd134f5-c817-4f6f-b861-b0d9562b90ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a3ed8bea97e49628311cb45d48ca699'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:50.988956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:50.993628 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-7fd134f5-c817-4f6f-b861-b0d9562b90ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a3ed8bea97e49628311cb45d48ca699'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:51.017684 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1285/2574] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:19:50 2022] DELETE /identity/v3/users/0a3ed8bea97e49628311cb45d48ca699 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:19:51.262959 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fb28e-7ec4-4643-b810-c86f928b99ff service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:51.284806 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74fb28e-7ec4-4643-b810-c86f928b99ff service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:51.344641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:51.358206 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562e819-e908-47ee-b7fb-748f9d2836e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:51.359792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562e819-e908-47ee-b7fb-748f9d2836e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:51.360057 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562e819-e908-47ee-b7fb-748f9d2836e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:51.360697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562e819-e908-47ee-b7fb-748f9d2836e7 admin admin] PATH_INFO: `/v3/projects/6476016c570542d3b00f9ed1507655da` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:51.362316 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562e819-e908-47ee-b7fb-748f9d2836e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6476016c570542d3b00f9ed1507655da)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:51.364324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562e819-e908-47ee-b7fb-748f9d2836e7 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:51.401744 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd134f5-c817-4f6f-b861-b0d9562b90ff admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:51.403577 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-7fd134f5-c817-4f6f-b861-b0d9562b90ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:51.419602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd134f5-c817-4f6f-b861-b0d9562b90ff admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:51.427090 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0562e819-e908-47ee-b7fb-748f9d2836e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6476016c570542d3b00f9ed1507655da', 'request_id': 'req-0562e819-e908-47ee-b7fb-748f9d2836e7'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:51.434550 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0562e819-e908-47ee-b7fb-748f9d2836e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6476016c570542d3b00f9ed1507655da', 'request_id': 'req-0562e819-e908-47ee-b7fb-748f9d2836e7'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:51.442117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0562e819-e908-47ee-b7fb-748f9d2836e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6476016c570542d3b00f9ed1507655da', 'request_id': 'req-0562e819-e908-47ee-b7fb-748f9d2836e7'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:51.456168 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0562e819-e908-47ee-b7fb-748f9d2836e7 admin admin] The token cache is being invalidate because project 6476016c570542d3b00f9ed1507655da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:19:51.456461 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0562e819-e908-47ee-b7fb-748f9d2836e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:51.457097 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-0562e819-e908-47ee-b7fb-748f9d2836e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:51.458801 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1290/2575] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:19:51 2022] DELETE /identity/v3/projects/6476016c570542d3b00f9ed1507655da => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:51.463538 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea8b53-f5af-48a2-a95e-f45df75fbe4a None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:51.464132 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea8b53-f5af-48a2-a95e-f45df75fbe4a None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:51.464240 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea8b53-f5af-48a2-a95e-f45df75fbe4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:51.482115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808ccc5-97f7-4663-8933-5f5f5fa40ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:51.482562 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-54ea8b53-f5af-48a2-a95e-f45df75fbe4a None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:51.483034 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808ccc5-97f7-4663-8933-5f5f5fa40ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:51.483328 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808ccc5-97f7-4663-8933-5f5f5fa40ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:51.483583 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808ccc5-97f7-4663-8933-5f5f5fa40ddf admin admin] PATH_INFO: `/v3/users/7220994b9eed4aa680dfa2ee1496d70f` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:51.483822 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-54ea8b53-f5af-48a2-a95e-f45df75fbe4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:51.485673 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808ccc5-97f7-4663-8933-5f5f5fa40ddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7220994b9eed4aa680dfa2ee1496d70f)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:51.487619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808ccc5-97f7-4663-8933-5f5f5fa40ddf admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:51.495093 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1291/2576] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:19:51.503134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a578274-11d9-41e1-ad20-4e1d8887a40c None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:51.503423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a578274-11d9-41e1-ad20-4e1d8887a40c None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:51.503701 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a578274-11d9-41e1-ad20-4e1d8887a40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:51.525948 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-8a578274-11d9-41e1-ad20-4e1d8887a40c None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:51.527182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8a578274-11d9-41e1-ad20-4e1d8887a40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:51.535736 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1292/2577] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:19:51.539872 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a578274-11d9-41e1-ad20-4e1d8887a40c None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:51.543417 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d808ccc5-97f7-4663-8933-5f5f5fa40ddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7220994b9eed4aa680dfa2ee1496d70f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:51.550483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a578274-11d9-41e1-ad20-4e1d8887a40c None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:51.555370 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce855c-4e11-4f68-87de-9ecff0efb1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:51.555969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce855c-4e11-4f68-87de-9ecff0efb1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:51.556296 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce855c-4e11-4f68-87de-9ecff0efb1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:51.556523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce855c-4e11-4f68-87de-9ecff0efb1b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:51.557150 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d808ccc5-97f7-4663-8933-5f5f5fa40ddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7220994b9eed4aa680dfa2ee1496d70f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:51.557282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce855c-4e11-4f68-87de-9ecff0efb1b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:19:51.557549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce855c-4e11-4f68-87de-9ecff0efb1b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:51.559730 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce855c-4e11-4f68-87de-9ecff0efb1b4 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:51.565391 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d808ccc5-97f7-4663-8933-5f5f5fa40ddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7220994b9eed4aa680dfa2ee1496d70f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:51.566105 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d808ccc5-97f7-4663-8933-5f5f5fa40ddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7220994b9eed4aa680dfa2ee1496d70f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:51.572525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1293/2578] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:19:51.574758 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d808ccc5-97f7-4663-8933-5f5f5fa40ddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7220994b9eed4aa680dfa2ee1496d70f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:51.577277 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce855c-4e11-4f68-87de-9ecff0efb1b4 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:51.579077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c0ce855c-4e11-4f68-87de-9ecff0efb1b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:51.582073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1286/2579] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:19:51 2022] DELETE /identity/v3/users/7220994b9eed4aa680dfa2ee1496d70f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:51.586138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a1936-ff40-4079-bdcc-02380651b4b1 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:51.586437 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a1936-ff40-4079-bdcc-02380651b4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:51.586932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a1936-ff40-4079-bdcc-02380651b4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:51.591449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce855c-4e11-4f68-87de-9ecff0efb1b4 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:51.604339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-db8a1936-ff40-4079-bdcc-02380651b4b1 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:51.605533 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-db8a1936-ff40-4079-bdcc-02380651b4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:51.652070 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bc1b2-3bb6-4dca-8b14-e0edccca6173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:51.653391 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bc1b2-3bb6-4dca-8b14-e0edccca6173 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:51.653709 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bc1b2-3bb6-4dca-8b14-e0edccca6173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:51.653993 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bc1b2-3bb6-4dca-8b14-e0edccca6173 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:51.654716 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6bc1b2-3bb6-4dca-8b14-e0edccca6173 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:51.657400 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6bc1b2-3bb6-4dca-8b14-e0edccca6173 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:51.668809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1287/2580] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:19:51.681911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a1936-ff40-4079-bdcc-02380651b4b1 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:51.683572 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-db8a1936-ff40-4079-bdcc-02380651b4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:51.688392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1294/2581] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:19:51 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:51.693105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bc1b2-3bb6-4dca-8b14-e0edccca6173 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:51.708026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:51.767764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:51.785870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a1936-ff40-4079-bdcc-02380651b4b1 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:51.807667 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bc1b2-3bb6-4dca-8b14-e0edccca6173 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:51.813583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fe86e-d9c8-4bb5-af8e-a14f32f270a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:51.814501 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fe86e-d9c8-4bb5-af8e-a14f32f270a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:51.814985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fe86e-d9c8-4bb5-af8e-a14f32f270a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:51.815767 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fe86e-d9c8-4bb5-af8e-a14f32f270a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:51.817385 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fe86e-d9c8-4bb5-af8e-a14f32f270a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:51.819642 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fe86e-d9c8-4bb5-af8e-a14f32f270a6 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:51.868210 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1295/2582] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:19:51 2022] POST /identity/v3/users => generated 291 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:51.872279 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fe86e-d9c8-4bb5-af8e-a14f32f270a6 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:51.879519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060c5d0-b44b-446e-ae5a-21ecbe2a8dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:51.881067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060c5d0-b44b-446e-ae5a-21ecbe2a8dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:51.881352 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060c5d0-b44b-446e-ae5a-21ecbe2a8dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:51.881769 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060c5d0-b44b-446e-ae5a-21ecbe2a8dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:51.894403 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060c5d0-b44b-446e-ae5a-21ecbe2a8dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:51.895838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060c5d0-b44b-446e-ae5a-21ecbe2a8dd4 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:51.901448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fe86e-d9c8-4bb5-af8e-a14f32f270a6 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:51.908689 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1288/2583] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:19:51.917414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcec339-3bba-43a0-ad30-b7d3e1237c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:51.918089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcec339-3bba-43a0-ad30-b7d3e1237c27 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:51.918382 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcec339-3bba-43a0-ad30-b7d3e1237c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:51.918665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcec339-3bba-43a0-ad30-b7d3e1237c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:51.919486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcec339-3bba-43a0-ad30-b7d3e1237c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:51.921099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcec339-3bba-43a0-ad30-b7d3e1237c27 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:51.936898 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1296/2584] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:19:51.941255 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060c5d0-b44b-446e-ae5a-21ecbe2a8dd4 service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:51.951338 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060c5d0-b44b-446e-ae5a-21ecbe2a8dd4 service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:51.957835 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710927db-cde3-487b-8510-26bd95d3ecb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:51.958650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710927db-cde3-487b-8510-26bd95d3ecb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:51.959134 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710927db-cde3-487b-8510-26bd95d3ecb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:51.959545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710927db-cde3-487b-8510-26bd95d3ecb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:51.960416 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710927db-cde3-487b-8510-26bd95d3ecb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:51.962231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710927db-cde3-487b-8510-26bd95d3ecb4 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:51.982721 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1289/2585] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:51.984432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcec339-3bba-43a0-ad30-b7d3e1237c27 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:51.999701 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcec339-3bba-43a0-ad30-b7d3e1237c27 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:52.007488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0ccef-fd71-408c-beac-23b8474ca730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:52.008319 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0ccef-fd71-408c-beac-23b8474ca730 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:52.008933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0ccef-fd71-408c-beac-23b8474ca730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:52.009169 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0ccef-fd71-408c-beac-23b8474ca730 admin admin] PATH_INFO: `/v3/projects/01915c889ff44ae2868559d18f5a3714/users/69389036be654235b7aa84c750d880cd/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:52.031549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf0ccef-fd71-408c-beac-23b8474ca730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01915c889ff44ae2868559d18f5a3714, user_id=69389036be654235b7aa84c750d880cd, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:52.034432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf0ccef-fd71-408c-beac-23b8474ca730 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:52.051112 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1297/2586] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:51 2022] PUT /identity/v3/projects/01915c889ff44ae2868559d18f5a3714/users/69389036be654235b7aa84c750d880cd/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:52.061345 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ee27f-b8ff-4387-b184-7aeaf2d41b05 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:52.061890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ee27f-b8ff-4387-b184-7aeaf2d41b05 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:52.062947 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51ee27f-b8ff-4387-b184-7aeaf2d41b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:52.100750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-c51ee27f-b8ff-4387-b184-7aeaf2d41b05 None None] MFA Rules not processed for user `69389036be654235b7aa84c750d880cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:52.102163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c51ee27f-b8ff-4387-b184-7aeaf2d41b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:52.175477 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1290/2587] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:52.186735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0ccef-fd71-408c-beac-23b8474ca730 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:52.189036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2bf0ccef-fd71-408c-beac-23b8474ca730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:52.285992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0ccef-fd71-408c-beac-23b8474ca730 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:52.293401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad7894-a3dd-44aa-af98-e6cd8b3d8d36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:52.294381 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad7894-a3dd-44aa-af98-e6cd8b3d8d36 service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:52.294896 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad7894-a3dd-44aa-af98-e6cd8b3d8d36 service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:52.295363 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad7894-a3dd-44aa-af98-e6cd8b3d8d36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:52.307623 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad7894-a3dd-44aa-af98-e6cd8b3d8d36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:52.309532 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad7894-a3dd-44aa-af98-e6cd8b3d8d36 service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:52.321542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1298/2588] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:52.423825 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ee27f-b8ff-4387-b184-7aeaf2d41b05 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:52.425477 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c51ee27f-b8ff-4387-b184-7aeaf2d41b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:52.527944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=100 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:52.558067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51ee27f-b8ff-4387-b184-7aeaf2d41b05 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:52.565739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd10e92-7c04-4afe-8545-cae8c789b9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:52.566618 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd10e92-7c04-4afe-8545-cae8c789b9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:52.566926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd10e92-7c04-4afe-8545-cae8c789b9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:52.567253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd10e92-7c04-4afe-8545-cae8c789b9d5 admin admin] PATH_INFO: `/v3/projects/3a8659d2936746b2a89772c02e01e248` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:52.569755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd10e92-7c04-4afe-8545-cae8c789b9d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a8659d2936746b2a89772c02e01e248)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:52.571704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd10e92-7c04-4afe-8545-cae8c789b9d5 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:52.671454 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4fd10e92-7c04-4afe-8545-cae8c789b9d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a8659d2936746b2a89772c02e01e248', 'request_id': 'req-4fd10e92-7c04-4afe-8545-cae8c789b9d5'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:52.677664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4fd10e92-7c04-4afe-8545-cae8c789b9d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a8659d2936746b2a89772c02e01e248', 'request_id': 'req-4fd10e92-7c04-4afe-8545-cae8c789b9d5'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:52.688085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4fd10e92-7c04-4afe-8545-cae8c789b9d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a8659d2936746b2a89772c02e01e248', 'request_id': 'req-4fd10e92-7c04-4afe-8545-cae8c789b9d5'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:52.717910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4fd10e92-7c04-4afe-8545-cae8c789b9d5 admin admin] The token cache is being invalidate because project 3a8659d2936746b2a89772c02e01e248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:19:52.718290 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4fd10e92-7c04-4afe-8545-cae8c789b9d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:52.719611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4fd10e92-7c04-4afe-8545-cae8c789b9d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:52.721533 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1291/2589] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:19:52 2022] DELETE /identity/v3/projects/3a8659d2936746b2a89772c02e01e248 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:19:52.726150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad7894-a3dd-44aa-af98-e6cd8b3d8d36 service nova] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:52.743343 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad7894-a3dd-44aa-af98-e6cd8b3d8d36 service nova] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:52.825317 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d3f07-4b07-432b-900f-333b6b059cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:52.826431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d3f07-4b07-432b-900f-333b6b059cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:52.826745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d3f07-4b07-432b-900f-333b6b059cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:52.827035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d3f07-4b07-432b-900f-333b6b059cb5 admin admin] PATH_INFO: `/v3/users/eb5e37a89dfd4911a0af70f7feb95fbb` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:52.829236 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d3f07-4b07-432b-900f-333b6b059cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb5e37a89dfd4911a0af70f7feb95fbb)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:52.831473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d3f07-4b07-432b-900f-333b6b059cb5 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:52.870036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:52.904262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-ae9d3f07-4b07-432b-900f-333b6b059cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb5e37a89dfd4911a0af70f7feb95fbb'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:52.924359 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-ae9d3f07-4b07-432b-900f-333b6b059cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb5e37a89dfd4911a0af70f7feb95fbb'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:52.933086 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-ae9d3f07-4b07-432b-900f-333b6b059cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb5e37a89dfd4911a0af70f7feb95fbb'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:52.933915 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-ae9d3f07-4b07-432b-900f-333b6b059cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb5e37a89dfd4911a0af70f7feb95fbb'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:52.943222 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-ae9d3f07-4b07-432b-900f-333b6b059cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb5e37a89dfd4911a0af70f7feb95fbb'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:52.951601 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1299/2590] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:19:52 2022] DELETE /identity/v3/users/eb5e37a89dfd4911a0af70f7feb95fbb => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:53.668744 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd10e92-7c04-4afe-8545-cae8c789b9d5 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:53.670384 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4fd10e92-7c04-4afe-8545-cae8c789b9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:53.684961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd10e92-7c04-4afe-8545-cae8c789b9d5 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:53.765697 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b240d46-4c24-40f8-b836-1e86ca24884f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:53.767980 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b240d46-4c24-40f8-b836-1e86ca24884f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:53.768381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b240d46-4c24-40f8-b836-1e86ca24884f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:53.769381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b240d46-4c24-40f8-b836-1e86ca24884f admin admin] PATH_INFO: `/v3/projects/908e393471644625b06e99ae26d95ebf` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:53.772840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b240d46-4c24-40f8-b836-1e86ca24884f admin admin] RBAC: Authorizing `identity:delete_project(project_id=908e393471644625b06e99ae26d95ebf)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:53.775170 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b240d46-4c24-40f8-b836-1e86ca24884f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:53.852357 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0b240d46-4c24-40f8-b836-1e86ca24884f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '908e393471644625b06e99ae26d95ebf', 'request_id': 'req-0b240d46-4c24-40f8-b836-1e86ca24884f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:53.859791 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0b240d46-4c24-40f8-b836-1e86ca24884f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '908e393471644625b06e99ae26d95ebf', 'request_id': 'req-0b240d46-4c24-40f8-b836-1e86ca24884f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:53.871893 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0b240d46-4c24-40f8-b836-1e86ca24884f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '908e393471644625b06e99ae26d95ebf', 'request_id': 'req-0b240d46-4c24-40f8-b836-1e86ca24884f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:53.902424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0b240d46-4c24-40f8-b836-1e86ca24884f admin admin] The token cache is being invalidate because project 908e393471644625b06e99ae26d95ebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:19:53.902971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0b240d46-4c24-40f8-b836-1e86ca24884f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:53.903728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-0b240d46-4c24-40f8-b836-1e86ca24884f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:53.905565 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1292/2591] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:19:53 2022] DELETE /identity/v3/projects/908e393471644625b06e99ae26d95ebf => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:53.916896 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82d210-4c26-4c4d-8a68-3a4db37bbc1e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:53.917495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82d210-4c26-4c4d-8a68-3a4db37bbc1e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:53.918073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82d210-4c26-4c4d-8a68-3a4db37bbc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:53.955758 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-9e82d210-4c26-4c4d-8a68-3a4db37bbc1e None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:53.957792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9e82d210-4c26-4c4d-8a68-3a4db37bbc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:54.122732 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1300/2592] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:54.127115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8dee9-f9c7-4b3c-bdb0-136ab23c9ebf None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:54.127587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8dee9-f9c7-4b3c-bdb0-136ab23c9ebf None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:54.128002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8dee9-f9c7-4b3c-bdb0-136ab23c9ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:54.163105 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-0eb8dee9-f9c7-4b3c-bdb0-136ab23c9ebf None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:54.164470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0eb8dee9-f9c7-4b3c-bdb0-136ab23c9ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:54.197349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82d210-4c26-4c4d-8a68-3a4db37bbc1e None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:54.199285 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1293/2593] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:54.214194 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82d210-4c26-4c4d-8a68-3a4db37bbc1e None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:54.235392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1649f5-aa8e-438f-a0f7-1fb95e9011cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:54.236304 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1649f5-aa8e-438f-a0f7-1fb95e9011cd admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:54.237777 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1649f5-aa8e-438f-a0f7-1fb95e9011cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:54.238076 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1649f5-aa8e-438f-a0f7-1fb95e9011cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:54.239373 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1649f5-aa8e-438f-a0f7-1fb95e9011cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:19:54.239755 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1649f5-aa8e-438f-a0f7-1fb95e9011cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:54.241548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1649f5-aa8e-438f-a0f7-1fb95e9011cd admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:54.266892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8dee9-f9c7-4b3c-bdb0-136ab23c9ebf None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:54.271462 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1301/2594] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:19:54.277094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8dee9-f9c7-4b3c-bdb0-136ab23c9ebf None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:54.294256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc63468-2759-41fb-8286-8e1d6ad57289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:54.296919 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc63468-2759-41fb-8286-8e1d6ad57289 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:54.297366 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc63468-2759-41fb-8286-8e1d6ad57289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:54.297654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc63468-2759-41fb-8286-8e1d6ad57289 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:54.300222 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc63468-2759-41fb-8286-8e1d6ad57289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:54.305284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc63468-2759-41fb-8286-8e1d6ad57289 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:54.311221 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:54.364372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1294/2595] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:19:54 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:54.368448 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1649f5-aa8e-438f-a0f7-1fb95e9011cd admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:54.382305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1649f5-aa8e-438f-a0f7-1fb95e9011cd admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:54.487964 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9714f7-2e8f-4fd7-a33b-8e3118d36889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:54.490134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9714f7-2e8f-4fd7-a33b-8e3118d36889 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:54.490427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9714f7-2e8f-4fd7-a33b-8e3118d36889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:54.490813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9714f7-2e8f-4fd7-a33b-8e3118d36889 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:54.492159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9714f7-2e8f-4fd7-a33b-8e3118d36889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:54.494248 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9714f7-2e8f-4fd7-a33b-8e3118d36889 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:54.528597 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1302/2596] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:19:54 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:54.533524 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc63468-2759-41fb-8286-8e1d6ad57289 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:54.544647 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc63468-2759-41fb-8286-8e1d6ad57289 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:54.555850 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91801182-cf04-498f-b72e-43f6c59f381b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:54.556682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91801182-cf04-498f-b72e-43f6c59f381b admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:54.557062 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91801182-cf04-498f-b72e-43f6c59f381b admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:54.557639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91801182-cf04-498f-b72e-43f6c59f381b admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:54.558437 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91801182-cf04-498f-b72e-43f6c59f381b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:54.561119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91801182-cf04-498f-b72e-43f6c59f381b admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:54.576094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1295/2597] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:54.580199 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9714f7-2e8f-4fd7-a33b-8e3118d36889 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:54.591643 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9714f7-2e8f-4fd7-a33b-8e3118d36889 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:54.597743 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b389ce0-be61-4cc4-8958-71ed87b5f356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:54.598315 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b389ce0-be61-4cc4-8958-71ed87b5f356 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:54.598662 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b389ce0-be61-4cc4-8958-71ed87b5f356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:54.599527 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b389ce0-be61-4cc4-8958-71ed87b5f356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:54.599761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b389ce0-be61-4cc4-8958-71ed87b5f356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:54.601404 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b389ce0-be61-4cc4-8958-71ed87b5f356 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:54.618943 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1303/2598] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:54.622992 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91801182-cf04-498f-b72e-43f6c59f381b admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:54.634599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91801182-cf04-498f-b72e-43f6c59f381b admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:54.639356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95314d9-93cd-4297-b28e-d79511faa220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:54.640484 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95314d9-93cd-4297-b28e-d79511faa220 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:54.640828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95314d9-93cd-4297-b28e-d79511faa220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:54.641165 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95314d9-93cd-4297-b28e-d79511faa220 admin admin] PATH_INFO: `/v3/projects/561446eabd144133bed76ba2291d392e/users/ff81dd44e702476ea9aab7eea7d82f1c/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:54.656883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95314d9-93cd-4297-b28e-d79511faa220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=561446eabd144133bed76ba2291d392e, user_id=ff81dd44e702476ea9aab7eea7d82f1c, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:54.659728 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95314d9-93cd-4297-b28e-d79511faa220 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:54.674085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1296/2599] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:54 2022] PUT /identity/v3/projects/561446eabd144133bed76ba2291d392e/users/ff81dd44e702476ea9aab7eea7d82f1c/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:54.684492 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30442f-d88a-4fd1-9dd7-f1619ff609f4 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:54.684853 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30442f-d88a-4fd1-9dd7-f1619ff609f4 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:54.685199 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30442f-d88a-4fd1-9dd7-f1619ff609f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:54.703930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:54.714509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-8a30442f-d88a-4fd1-9dd7-f1619ff609f4 None None] MFA Rules not processed for user `ff81dd44e702476ea9aab7eea7d82f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:54.716454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8a30442f-d88a-4fd1-9dd7-f1619ff609f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:54.780437 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1304/2600] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:54.786094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95314d9-93cd-4297-b28e-d79511faa220 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:54.848092 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95314d9-93cd-4297-b28e-d79511faa220 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:54.852614 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0007d8-819a-4829-8b8d-3b7fb6a8d2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:54.853266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0007d8-819a-4829-8b8d-3b7fb6a8d2bb admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:54.853549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0007d8-819a-4829-8b8d-3b7fb6a8d2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:54.853824 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0007d8-819a-4829-8b8d-3b7fb6a8d2bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:54.854514 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0007d8-819a-4829-8b8d-3b7fb6a8d2bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:54.856466 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0007d8-819a-4829-8b8d-3b7fb6a8d2bb admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:54.881258 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1297/2601] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:19:54 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:19:54.885110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30442f-d88a-4fd1-9dd7-f1619ff609f4 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:54.896352 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30442f-d88a-4fd1-9dd7-f1619ff609f4 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:54.951303 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebcfe6-fb1a-45ba-b753-8df76b1b2212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:54.951962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebcfe6-fb1a-45ba-b753-8df76b1b2212 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:54.952243 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebcfe6-fb1a-45ba-b753-8df76b1b2212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:54.953008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebcfe6-fb1a-45ba-b753-8df76b1b2212 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:54.953700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ebcfe6-fb1a-45ba-b753-8df76b1b2212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:54.955663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ebcfe6-fb1a-45ba-b753-8df76b1b2212 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:54.975940 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1305/2602] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:19:54 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:19:54.979865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0007d8-819a-4829-8b8d-3b7fb6a8d2bb admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:54.990182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0007d8-819a-4829-8b8d-3b7fb6a8d2bb admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:54.994838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed4c68-2d41-4004-a1fe-a6bcfe5406f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:54.995505 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed4c68-2d41-4004-a1fe-a6bcfe5406f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:54.995789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed4c68-2d41-4004-a1fe-a6bcfe5406f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:54.996071 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed4c68-2d41-4004-a1fe-a6bcfe5406f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:54.996801 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed4c68-2d41-4004-a1fe-a6bcfe5406f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:54.998381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed4c68-2d41-4004-a1fe-a6bcfe5406f9 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:55.008850 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1298/2603] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:19:55.012769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebcfe6-fb1a-45ba-b753-8df76b1b2212 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:55.023556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebcfe6-fb1a-45ba-b753-8df76b1b2212 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:55.028329 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c889b-cc27-4694-abc9-e2a0a6b1d3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:55.029214 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c889b-cc27-4694-abc9-e2a0a6b1d3cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:55.029501 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c889b-cc27-4694-abc9-e2a0a6b1d3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:55.029782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c889b-cc27-4694-abc9-e2a0a6b1d3cf admin admin] PATH_INFO: `/v3/projects/1262739df7864a03a8a2454c0b0f7285/users/34ea89f50eab41e4a3e11d784407b717/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:55.047561 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c889b-cc27-4694-abc9-e2a0a6b1d3cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1262739df7864a03a8a2454c0b0f7285, user_id=34ea89f50eab41e4a3e11d784407b717, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:55.050203 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c889b-cc27-4694-abc9-e2a0a6b1d3cf admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:55.062659 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1306/2604] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:55 2022] PUT /identity/v3/projects/1262739df7864a03a8a2454c0b0f7285/users/34ea89f50eab41e4a3e11d784407b717/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:19:55.066560 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed4c68-2d41-4004-a1fe-a6bcfe5406f9 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:55.076541 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed4c68-2d41-4004-a1fe-a6bcfe5406f9 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:55.131522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57ff71-7f93-4f26-8169-73430592541f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:55.132183 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57ff71-7f93-4f26-8169-73430592541f admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:55.132468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57ff71-7f93-4f26-8169-73430592541f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:55.132746 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57ff71-7f93-4f26-8169-73430592541f admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:55.133467 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c57ff71-7f93-4f26-8169-73430592541f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:55.135112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c57ff71-7f93-4f26-8169-73430592541f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:55.145086 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1299/2605] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:19:55.149151 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c889b-cc27-4694-abc9-e2a0a6b1d3cf admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:55.160226 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c889b-cc27-4694-abc9-e2a0a6b1d3cf admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:55.164838 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249eafe-4f12-4eae-ad12-15537c1b4c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:55.165708 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249eafe-4f12-4eae-ad12-15537c1b4c30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:55.165992 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249eafe-4f12-4eae-ad12-15537c1b4c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:55.166274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249eafe-4f12-4eae-ad12-15537c1b4c30 admin admin] PATH_INFO: `/v3/projects/1262739df7864a03a8a2454c0b0f7285/users/34ea89f50eab41e4a3e11d784407b717/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:55.168267 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249eafe-4f12-4eae-ad12-15537c1b4c30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1262739df7864a03a8a2454c0b0f7285, user_id=34ea89f50eab41e4a3e11d784407b717, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:55.170924 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249eafe-4f12-4eae-ad12-15537c1b4c30 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:55.180799 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1307/2606] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:19:55 2022] PUT /identity/v3/projects/1262739df7864a03a8a2454c0b0f7285/users/34ea89f50eab41e4a3e11d784407b717/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:55.191964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043d833-f82a-4c8d-bedf-b211b3ea4e02 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:55.192262 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043d833-f82a-4c8d-bedf-b211b3ea4e02 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:55.192551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043d833-f82a-4c8d-bedf-b211b3ea4e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:55.219487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a043d833-f82a-4c8d-bedf-b211b3ea4e02 None None] MFA Rules not processed for user `34ea89f50eab41e4a3e11d784407b717`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:19:55.220682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a043d833-f82a-4c8d-bedf-b211b3ea4e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:55.228920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:55.290020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1300/2607] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:19:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:19:55.300999 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249eafe-4f12-4eae-ad12-15537c1b4c30 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:55.302733 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d249eafe-4f12-4eae-ad12-15537c1b4c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:55.361719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249eafe-4f12-4eae-ad12-15537c1b4c30 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:55.365341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7042e6b-6ab6-4ec8-aabc-05f271497853 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:55.365989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7042e6b-6ab6-4ec8-aabc-05f271497853 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:55.366273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7042e6b-6ab6-4ec8-aabc-05f271497853 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:55.366556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7042e6b-6ab6-4ec8-aabc-05f271497853 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:55.370666 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:55.426613 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7042e6b-6ab6-4ec8-aabc-05f271497853 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:55.428018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7042e6b-6ab6-4ec8-aabc-05f271497853 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:55.437513 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1308/2608] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:19:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:19:59.283079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043d833-f82a-4c8d-bedf-b211b3ea4e02 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:59.284853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a043d833-f82a-4c8d-bedf-b211b3ea4e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:59.349126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043d833-f82a-4c8d-bedf-b211b3ea4e02 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:59.353682 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe46be-5486-41bc-9e54-66c5ed31f06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:59.354549 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe46be-5486-41bc-9e54-66c5ed31f06b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:59.354865 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe46be-5486-41bc-9e54-66c5ed31f06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:59.355146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe46be-5486-41bc-9e54-66c5ed31f06b admin admin] PATH_INFO: `/v3/users/7c367c44c73d4f2790e37ebaeb5c7192` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:59.356322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe46be-5486-41bc-9e54-66c5ed31f06b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c367c44c73d4f2790e37ebaeb5c7192)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:59.358241 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe46be-5486-41bc-9e54-66c5ed31f06b admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:59.404513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-7fbe46be-5486-41bc-9e54-66c5ed31f06b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c367c44c73d4f2790e37ebaeb5c7192'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:59.418085 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-7fbe46be-5486-41bc-9e54-66c5ed31f06b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c367c44c73d4f2790e37ebaeb5c7192'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:59.424944 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-7fbe46be-5486-41bc-9e54-66c5ed31f06b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c367c44c73d4f2790e37ebaeb5c7192'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:59.425634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-7fbe46be-5486-41bc-9e54-66c5ed31f06b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c367c44c73d4f2790e37ebaeb5c7192'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:59.432408 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-7fbe46be-5486-41bc-9e54-66c5ed31f06b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c367c44c73d4f2790e37ebaeb5c7192'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:59.438474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1301/2609] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:19:59 2022] DELETE /identity/v3/users/7c367c44c73d4f2790e37ebaeb5c7192 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:19:59.606804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7042e6b-6ab6-4ec8-aabc-05f271497853 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:59.618353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7042e6b-6ab6-4ec8-aabc-05f271497853 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:59.676284 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b2d61-285a-468c-8e36-24a9e307aa07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:59.677148 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b2d61-285a-468c-8e36-24a9e307aa07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:59.677433 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b2d61-285a-468c-8e36-24a9e307aa07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:59.677716 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b2d61-285a-468c-8e36-24a9e307aa07 admin admin] PATH_INFO: `/v3/projects/44bf06927ad44c4b88d54f786781be68` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:59.679051 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b2d61-285a-468c-8e36-24a9e307aa07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44bf06927ad44c4b88d54f786781be68)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:59.680999 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b2d61-285a-468c-8e36-24a9e307aa07 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:59.726633 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-153b2d61-285a-468c-8e36-24a9e307aa07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44bf06927ad44c4b88d54f786781be68', 'request_id': 'req-153b2d61-285a-468c-8e36-24a9e307aa07'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:59.732097 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-153b2d61-285a-468c-8e36-24a9e307aa07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44bf06927ad44c4b88d54f786781be68', 'request_id': 'req-153b2d61-285a-468c-8e36-24a9e307aa07'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:59.738087 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-153b2d61-285a-468c-8e36-24a9e307aa07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44bf06927ad44c4b88d54f786781be68', 'request_id': 'req-153b2d61-285a-468c-8e36-24a9e307aa07'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:59.750355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-153b2d61-285a-468c-8e36-24a9e307aa07 admin admin] The token cache is being invalidate because project 44bf06927ad44c4b88d54f786781be68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:19:59.750646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-153b2d61-285a-468c-8e36-24a9e307aa07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:59.751309 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-153b2d61-285a-468c-8e36-24a9e307aa07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:59.752880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1309/2610] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:19:59 2022] DELETE /identity/v3/projects/44bf06927ad44c4b88d54f786781be68 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:19:59.756923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe46be-5486-41bc-9e54-66c5ed31f06b admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:19:59.758280 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-7fbe46be-5486-41bc-9e54-66c5ed31f06b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:19:59.771686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe46be-5486-41bc-9e54-66c5ed31f06b admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:19:59.790818 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:19:59.829902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e7d6e-ed06-44dc-9d40-62efe2adb92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:19:59.830819 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e7d6e-ed06-44dc-9d40-62efe2adb92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:19:59.831107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e7d6e-ed06-44dc-9d40-62efe2adb92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:19:59.831387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e7d6e-ed06-44dc-9d40-62efe2adb92c admin admin] PATH_INFO: `/v3/users/4ccfc1a5399c44459213698977990536` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:19:59.832607 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e7d6e-ed06-44dc-9d40-62efe2adb92c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ccfc1a5399c44459213698977990536)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:19:59.834499 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e7d6e-ed06-44dc-9d40-62efe2adb92c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:19:59.880650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-df9e7d6e-ed06-44dc-9d40-62efe2adb92c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ccfc1a5399c44459213698977990536'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:59.893204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-df9e7d6e-ed06-44dc-9d40-62efe2adb92c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ccfc1a5399c44459213698977990536'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:59.900008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-df9e7d6e-ed06-44dc-9d40-62efe2adb92c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ccfc1a5399c44459213698977990536'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:59.900684 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-df9e7d6e-ed06-44dc-9d40-62efe2adb92c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ccfc1a5399c44459213698977990536'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:59.907396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-df9e7d6e-ed06-44dc-9d40-62efe2adb92c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ccfc1a5399c44459213698977990536'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:19:59.913307 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1302/2611] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:19:59 2022] DELETE /identity/v3/users/4ccfc1a5399c44459213698977990536 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:00.084844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b2d61-285a-468c-8e36-24a9e307aa07 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:00.086395 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-153b2d61-285a-468c-8e36-24a9e307aa07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:00.101794 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b2d61-285a-468c-8e36-24a9e307aa07 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:00.137923 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:00.165139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bbf30-92a5-4e0d-9af3-3952fd446916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:00.166033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bbf30-92a5-4e0d-9af3-3952fd446916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:00.166316 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bbf30-92a5-4e0d-9af3-3952fd446916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:00.166589 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bbf30-92a5-4e0d-9af3-3952fd446916 admin admin] PATH_INFO: `/v3/projects/0c7042d786744a53a1b9f0ef7009dc46` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:00.167793 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bbf30-92a5-4e0d-9af3-3952fd446916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c7042d786744a53a1b9f0ef7009dc46)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:00.169674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bbf30-92a5-4e0d-9af3-3952fd446916 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:00.217230 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-972bbf30-92a5-4e0d-9af3-3952fd446916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c7042d786744a53a1b9f0ef7009dc46', 'request_id': 'req-972bbf30-92a5-4e0d-9af3-3952fd446916'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:00.223590 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-972bbf30-92a5-4e0d-9af3-3952fd446916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c7042d786744a53a1b9f0ef7009dc46', 'request_id': 'req-972bbf30-92a5-4e0d-9af3-3952fd446916'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:00.230117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-972bbf30-92a5-4e0d-9af3-3952fd446916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c7042d786744a53a1b9f0ef7009dc46', 'request_id': 'req-972bbf30-92a5-4e0d-9af3-3952fd446916'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:00.242783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-972bbf30-92a5-4e0d-9af3-3952fd446916 admin admin] The token cache is being invalidate because project 0c7042d786744a53a1b9f0ef7009dc46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:00.243073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-972bbf30-92a5-4e0d-9af3-3952fd446916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:00.243698 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-972bbf30-92a5-4e0d-9af3-3952fd446916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:00.245254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1310/2612] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:00 2022] DELETE /identity/v3/projects/0c7042d786744a53a1b9f0ef7009dc46 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:00.250142 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7e5fb-d973-4410-8fc1-f8e679d54150 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:00.250431 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7e5fb-d973-4410-8fc1-f8e679d54150 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:00.250760 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7e5fb-d973-4410-8fc1-f8e679d54150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:00.269466 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-35b7e5fb-d973-4410-8fc1-f8e679d54150 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:00.270600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-35b7e5fb-d973-4410-8fc1-f8e679d54150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:00.331319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1303/2613] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:00.339355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5d539-3267-4536-8c17-df8c1a50840d None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:00.339646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5d539-3267-4536-8c17-df8c1a50840d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:00.339929 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5d539-3267-4536-8c17-df8c1a50840d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:00.359660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-14b5d539-3267-4536-8c17-df8c1a50840d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:00.360888 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-14b5d539-3267-4536-8c17-df8c1a50840d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:00.369642 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1311/2614] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:00.373890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7e5fb-d973-4410-8fc1-f8e679d54150 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:00.384506 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7e5fb-d973-4410-8fc1-f8e679d54150 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:00.389109 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ba462-66e7-4ff8-9ba2-7ab7917dd753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:00.389772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ba462-66e7-4ff8-9ba2-7ab7917dd753 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:00.390580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ba462-66e7-4ff8-9ba2-7ab7917dd753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:00.390889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ba462-66e7-4ff8-9ba2-7ab7917dd753 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:00.391622 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ba462-66e7-4ff8-9ba2-7ab7917dd753 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:20:00.391934 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ba462-66e7-4ff8-9ba2-7ab7917dd753 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:00.393529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ba462-66e7-4ff8-9ba2-7ab7917dd753 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:00.406369 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1304/2615] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:20:00.410908 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5d539-3267-4536-8c17-df8c1a50840d None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:00.420917 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5d539-3267-4536-8c17-df8c1a50840d None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:00.426537 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b2f56-f93b-4ee7-a661-ec5c623cd5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:00.427229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b2f56-f93b-4ee7-a661-ec5c623cd5a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:00.427516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b2f56-f93b-4ee7-a661-ec5c623cd5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:00.427791 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b2f56-f93b-4ee7-a661-ec5c623cd5a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:00.428494 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b2f56-f93b-4ee7-a661-ec5c623cd5a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:00.430389 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b2f56-f93b-4ee7-a661-ec5c623cd5a2 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:00.457240 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1312/2616] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:20:00 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:00.461233 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ba462-66e7-4ff8-9ba2-7ab7917dd753 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:00.471223 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ba462-66e7-4ff8-9ba2-7ab7917dd753 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:00.500941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:00.530726 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cbbfb-c4b4-42eb-8092-244d91130941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:00.531325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cbbfb-c4b4-42eb-8092-244d91130941 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:00.531608 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cbbfb-c4b4-42eb-8092-244d91130941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:00.531885 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cbbfb-c4b4-42eb-8092-244d91130941 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:00.532569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cbbfb-c4b4-42eb-8092-244d91130941 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:00.534467 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cbbfb-c4b4-42eb-8092-244d91130941 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:00.553191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1305/2617] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:20:00 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:00.557354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b2f56-f93b-4ee7-a661-ec5c623cd5a2 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:00.568270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b2f56-f93b-4ee7-a661-ec5c623cd5a2 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:00.572807 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d81677-c130-4168-837e-f436bcc89330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:00.573459 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d81677-c130-4168-837e-f436bcc89330 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:00.573742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d81677-c130-4168-837e-f436bcc89330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:00.574021 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d81677-c130-4168-837e-f436bcc89330 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:00.574775 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d81677-c130-4168-837e-f436bcc89330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:00.576600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d81677-c130-4168-837e-f436bcc89330 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:00.587567 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1313/2618] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:20:00.591349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cbbfb-c4b4-42eb-8092-244d91130941 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:00.601400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cbbfb-c4b4-42eb-8092-244d91130941 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:00.605928 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bb30e-1489-49a6-9d22-01150f3c7be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:00.606576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bb30e-1489-49a6-9d22-01150f3c7be8 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:00.606903 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bb30e-1489-49a6-9d22-01150f3c7be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:00.607184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bb30e-1489-49a6-9d22-01150f3c7be8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:00.607898 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bb30e-1489-49a6-9d22-01150f3c7be8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:00.609509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bb30e-1489-49a6-9d22-01150f3c7be8 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:00.619192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1306/2619] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:00.623297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d81677-c130-4168-837e-f436bcc89330 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:00.633716 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d81677-c130-4168-837e-f436bcc89330 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:00.638427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f8b96-aa39-4026-83b0-fda235b5b18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:00.639360 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f8b96-aa39-4026-83b0-fda235b5b18a admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:00.639642 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f8b96-aa39-4026-83b0-fda235b5b18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:00.640159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f8b96-aa39-4026-83b0-fda235b5b18a admin admin] PATH_INFO: `/v3/projects/93bdfaf479ca496fb687173ef2cb5b5a/users/a7fa96d72da44c4bad9713076d7c96fa/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:00.654610 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f8b96-aa39-4026-83b0-fda235b5b18a admin admin] RBAC: Authorizing `identity:create_grant(project_id=93bdfaf479ca496fb687173ef2cb5b5a, user_id=a7fa96d72da44c4bad9713076d7c96fa, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:00.657782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f8b96-aa39-4026-83b0-fda235b5b18a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:00.670245 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1314/2620] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:00 2022] PUT /identity/v3/projects/93bdfaf479ca496fb687173ef2cb5b5a/users/a7fa96d72da44c4bad9713076d7c96fa/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:00.682075 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45dd3e-7074-4874-a34d-38f86759cad0 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:00.682372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45dd3e-7074-4874-a34d-38f86759cad0 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:00.682657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45dd3e-7074-4874-a34d-38f86759cad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:00.708781 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-ca45dd3e-7074-4874-a34d-38f86759cad0 None None] MFA Rules not processed for user `a7fa96d72da44c4bad9713076d7c96fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:00.710043 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ca45dd3e-7074-4874-a34d-38f86759cad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:00.774294 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1307/2621] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:00.785948 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f8b96-aa39-4026-83b0-fda235b5b18a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:00.787618 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-1c0f8b96-aa39-4026-83b0-fda235b5b18a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:00.847454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f8b96-aa39-4026-83b0-fda235b5b18a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:00.852111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc092493-5855-447e-9e38-065b82822138 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb93eb1d25c4410591125b0ab7a73149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:00.853226 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc092493-5855-447e-9e38-065b82822138 service glance] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:00.853831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc092493-5855-447e-9e38-065b82822138 service glance] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:00.854355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc092493-5855-447e-9e38-065b82822138 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:00.867641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc092493-5855-447e-9e38-065b82822138 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:00.870514 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc092493-5855-447e-9e38-065b82822138 service glance] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:00.882720 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1315/2622] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:01.173510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45dd3e-7074-4874-a34d-38f86759cad0 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:01.175056 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-ca45dd3e-7074-4874-a34d-38f86759cad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:01.246356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45dd3e-7074-4874-a34d-38f86759cad0 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:01.251084 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52438a-76b8-4a25-af40-3587792a095d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:01.252868 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52438a-76b8-4a25-af40-3587792a095d service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:01.253211 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52438a-76b8-4a25-af40-3587792a095d service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:01.253570 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52438a-76b8-4a25-af40-3587792a095d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:01.255592 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6b52438a-76b8-4a25-af40-3587792a095d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:01.323146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:01.338371 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52438a-76b8-4a25-af40-3587792a095d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:01.340952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52438a-76b8-4a25-af40-3587792a095d service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:01.352191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1308/2623] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:01 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:01.433930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc092493-5855-447e-9e38-065b82822138 service glance] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:01.435604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-fc092493-5855-447e-9e38-065b82822138 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:01.442948 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:01.520207 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc092493-5855-447e-9e38-065b82822138 service glance] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:01.525255 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8c830-c4d7-4ee1-8472-2e88b4069080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:01.526135 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8c830-c4d7-4ee1-8472-2e88b4069080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:01.526422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8c830-c4d7-4ee1-8472-2e88b4069080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:01.526746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8c830-c4d7-4ee1-8472-2e88b4069080 admin admin] PATH_INFO: `/v3/users/69389036be654235b7aa84c750d880cd` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:01.528026 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8c830-c4d7-4ee1-8472-2e88b4069080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69389036be654235b7aa84c750d880cd)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:01.530013 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8c830-c4d7-4ee1-8472-2e88b4069080 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:01.619982 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d5a8c830-c4d7-4ee1-8472-2e88b4069080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69389036be654235b7aa84c750d880cd'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:01.637368 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d5a8c830-c4d7-4ee1-8472-2e88b4069080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69389036be654235b7aa84c750d880cd'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:01.648727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d5a8c830-c4d7-4ee1-8472-2e88b4069080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69389036be654235b7aa84c750d880cd'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:01.649600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d5a8c830-c4d7-4ee1-8472-2e88b4069080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69389036be654235b7aa84c750d880cd'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:01.673069 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d5a8c830-c4d7-4ee1-8472-2e88b4069080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69389036be654235b7aa84c750d880cd'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:01.680697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1316/2624] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:20:01 2022] DELETE /identity/v3/users/69389036be654235b7aa84c750d880cd => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:01.685493 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bdd66-7021-48d4-83f2-a1eac2e116b6 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:01.685806 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bdd66-7021-48d4-83f2-a1eac2e116b6 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:01.686143 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bdd66-7021-48d4-83f2-a1eac2e116b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:01.711984 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-6d2bdd66-7021-48d4-83f2-a1eac2e116b6 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:01.713221 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6d2bdd66-7021-48d4-83f2-a1eac2e116b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:01.782288 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1309/2625] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:20:01.791921 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8c830-c4d7-4ee1-8472-2e88b4069080 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:01.793353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d5a8c830-c4d7-4ee1-8472-2e88b4069080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:01.855099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8c830-c4d7-4ee1-8472-2e88b4069080 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:01.861070 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e4383-301f-4118-a739-cf628f9c7403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:01.861768 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e4383-301f-4118-a739-cf628f9c7403 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:01.862058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e4383-301f-4118-a739-cf628f9c7403 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:01.862341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e4383-301f-4118-a739-cf628f9c7403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:01.872361 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e4383-301f-4118-a739-cf628f9c7403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:01.873823 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e4383-301f-4118-a739-cf628f9c7403 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:01.885001 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1317/2626] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:02.176444 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bdd66-7021-48d4-83f2-a1eac2e116b6 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:02.177902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6d2bdd66-7021-48d4-83f2-a1eac2e116b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:02.190416 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bdd66-7021-48d4-83f2-a1eac2e116b6 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:02.195044 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec25e50-7026-46a2-b69b-7f1c6c666a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:02.195912 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec25e50-7026-46a2-b69b-7f1c6c666a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:02.196194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec25e50-7026-46a2-b69b-7f1c6c666a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:02.196469 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec25e50-7026-46a2-b69b-7f1c6c666a24 admin admin] PATH_INFO: `/v3/projects/01915c889ff44ae2868559d18f5a3714` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:02.197642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec25e50-7026-46a2-b69b-7f1c6c666a24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01915c889ff44ae2868559d18f5a3714)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:02.199589 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec25e50-7026-46a2-b69b-7f1c6c666a24 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:02.253810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-7ec25e50-7026-46a2-b69b-7f1c6c666a24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01915c889ff44ae2868559d18f5a3714', 'request_id': 'req-7ec25e50-7026-46a2-b69b-7f1c6c666a24'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:02.259306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-7ec25e50-7026-46a2-b69b-7f1c6c666a24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01915c889ff44ae2868559d18f5a3714', 'request_id': 'req-7ec25e50-7026-46a2-b69b-7f1c6c666a24'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:02.266535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-7ec25e50-7026-46a2-b69b-7f1c6c666a24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01915c889ff44ae2868559d18f5a3714', 'request_id': 'req-7ec25e50-7026-46a2-b69b-7f1c6c666a24'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:02.277427 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-7ec25e50-7026-46a2-b69b-7f1c6c666a24 admin admin] The token cache is being invalidate because project 01915c889ff44ae2868559d18f5a3714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:02.277714 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-7ec25e50-7026-46a2-b69b-7f1c6c666a24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:02.278324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-7ec25e50-7026-46a2-b69b-7f1c6c666a24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:02.281237 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1310/2627] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:02 2022] DELETE /identity/v3/projects/01915c889ff44ae2868559d18f5a3714 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:02.289093 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f3f06-6da9-42b9-8055-6aae52bcee43 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:02.289472 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f3f06-6da9-42b9-8055-6aae52bcee43 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:02.289751 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f3f06-6da9-42b9-8055-6aae52bcee43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:02.307483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-9f9f3f06-6da9-42b9-8055-6aae52bcee43 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:02.308671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9f9f3f06-6da9-42b9-8055-6aae52bcee43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:02.329395 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:02.374057 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1318/2628] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:02.383197 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048917a-ada8-40de-9a4a-c5cc47256ca2 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:02.383905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048917a-ada8-40de-9a4a-c5cc47256ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:02.384257 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048917a-ada8-40de-9a4a-c5cc47256ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:02.409055 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-f048917a-ada8-40de-9a4a-c5cc47256ca2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:02.410567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f048917a-ada8-40de-9a4a-c5cc47256ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:02.418979 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1311/2629] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:02.423265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f3f06-6da9-42b9-8055-6aae52bcee43 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:02.432840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f3f06-6da9-42b9-8055-6aae52bcee43 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:02.437469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7b38b-813d-4083-8c4a-2b4fc9a8d013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:02.438127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7b38b-813d-4083-8c4a-2b4fc9a8d013 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:02.438408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7b38b-813d-4083-8c4a-2b4fc9a8d013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:02.438753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7b38b-813d-4083-8c4a-2b4fc9a8d013 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:02.439524 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7b38b-813d-4083-8c4a-2b4fc9a8d013 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:20:02.439843 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7b38b-813d-4083-8c4a-2b4fc9a8d013 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:02.441362 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7b38b-813d-4083-8c4a-2b4fc9a8d013 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:02.454571 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1319/2630] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:20:02.459102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048917a-ada8-40de-9a4a-c5cc47256ca2 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:02.469311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048917a-ada8-40de-9a4a-c5cc47256ca2 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:02.474081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f69c5-92f0-45fb-b6ba-48218394cd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:02.474759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f69c5-92f0-45fb-b6ba-48218394cd15 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:02.475048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f69c5-92f0-45fb-b6ba-48218394cd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:02.475326 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f69c5-92f0-45fb-b6ba-48218394cd15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:02.476008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f69c5-92f0-45fb-b6ba-48218394cd15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:02.478105 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f69c5-92f0-45fb-b6ba-48218394cd15 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:02.506732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1312/2631] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:20:02 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:02.510661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7b38b-813d-4083-8c4a-2b4fc9a8d013 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:02.521084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7b38b-813d-4083-8c4a-2b4fc9a8d013 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:02.581110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61778746-1a71-43ef-809b-72dc05b6441b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:02.581792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61778746-1a71-43ef-809b-72dc05b6441b admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:02.582081 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61778746-1a71-43ef-809b-72dc05b6441b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:02.582361 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61778746-1a71-43ef-809b-72dc05b6441b admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:02.583089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61778746-1a71-43ef-809b-72dc05b6441b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:02.584986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61778746-1a71-43ef-809b-72dc05b6441b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:02.606706 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1320/2632] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:20:02 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:02.611059 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f69c5-92f0-45fb-b6ba-48218394cd15 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:02.621147 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f69c5-92f0-45fb-b6ba-48218394cd15 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:02.626322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5a416-9a2c-48a3-b651-a18e68d17166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:02.626993 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5a416-9a2c-48a3-b651-a18e68d17166 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:02.627282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5a416-9a2c-48a3-b651-a18e68d17166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:02.627563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5a416-9a2c-48a3-b651-a18e68d17166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:02.628301 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5a416-9a2c-48a3-b651-a18e68d17166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:02.629892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5a416-9a2c-48a3-b651-a18e68d17166 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:02.640912 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1313/2633] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:02.644877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61778746-1a71-43ef-809b-72dc05b6441b admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:02.654911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61778746-1a71-43ef-809b-72dc05b6441b admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:02.660212 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b81f9-3bb6-40a8-836b-3e56f7d9e23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:02.660880 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b81f9-3bb6-40a8-836b-3e56f7d9e23e admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:02.661170 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b81f9-3bb6-40a8-836b-3e56f7d9e23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:02.661455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b81f9-3bb6-40a8-836b-3e56f7d9e23e admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:02.662185 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b81f9-3bb6-40a8-836b-3e56f7d9e23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:02.663796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b81f9-3bb6-40a8-836b-3e56f7d9e23e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:02.674001 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1321/2634] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:02.682168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5a416-9a2c-48a3-b651-a18e68d17166 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:02.692763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5a416-9a2c-48a3-b651-a18e68d17166 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:02.697501 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db93a6-2ffd-4c61-b104-ee3c53b5188a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:02.698380 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db93a6-2ffd-4c61-b104-ee3c53b5188a admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:02.698657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db93a6-2ffd-4c61-b104-ee3c53b5188a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:02.699034 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db93a6-2ffd-4c61-b104-ee3c53b5188a admin admin] PATH_INFO: `/v3/projects/49dbfbad631a4c719daf89c42a7246bc/users/9dd4d112ea504fc991f4347eceb72183/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:02.714744 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db93a6-2ffd-4c61-b104-ee3c53b5188a admin admin] RBAC: Authorizing `identity:create_grant(project_id=49dbfbad631a4c719daf89c42a7246bc, user_id=9dd4d112ea504fc991f4347eceb72183, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:02.717633 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db93a6-2ffd-4c61-b104-ee3c53b5188a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:02.737611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1314/2635] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:02 2022] PUT /identity/v3/projects/49dbfbad631a4c719daf89c42a7246bc/users/9dd4d112ea504fc991f4347eceb72183/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:02.748159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fdaf4b-b95b-4735-9820-4cebd417568e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:02.748469 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fdaf4b-b95b-4735-9820-4cebd417568e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:02.748749 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fdaf4b-b95b-4735-9820-4cebd417568e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:02.787984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-05fdaf4b-b95b-4735-9820-4cebd417568e None None] MFA Rules not processed for user `9dd4d112ea504fc991f4347eceb72183`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:02.789345 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-05fdaf4b-b95b-4735-9820-4cebd417568e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:02.868697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1322/2636] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:02.872942 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db93a6-2ffd-4c61-b104-ee3c53b5188a admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:02.883694 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:02.954404 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db93a6-2ffd-4c61-b104-ee3c53b5188a admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:02.959174 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814102e5-4083-4bc0-bd63-0367fa8fa88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:02.959836 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814102e5-4083-4bc0-bd63-0367fa8fa88e admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:02.960152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814102e5-4083-4bc0-bd63-0367fa8fa88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:02.960434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814102e5-4083-4bc0-bd63-0367fa8fa88e admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:02.961133 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814102e5-4083-4bc0-bd63-0367fa8fa88e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:02.963076 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814102e5-4083-4bc0-bd63-0367fa8fa88e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:02.988224 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1315/2637] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:20:02 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:02.992740 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fdaf4b-b95b-4735-9820-4cebd417568e None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:03.004180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fdaf4b-b95b-4735-9820-4cebd417568e None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:03.022633 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:03.065324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60df86-261e-4f68-b7c9-ae5b9713dab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:03.065324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60df86-261e-4f68-b7c9-ae5b9713dab7 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:03.065324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60df86-261e-4f68-b7c9-ae5b9713dab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:03.065324 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60df86-261e-4f68-b7c9-ae5b9713dab7 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:03.065810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a60df86-261e-4f68-b7c9-ae5b9713dab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:03.067268 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a60df86-261e-4f68-b7c9-ae5b9713dab7 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:03.088639 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1323/2638] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:20:02 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:03.092599 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814102e5-4083-4bc0-bd63-0367fa8fa88e admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:03.104039 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814102e5-4083-4bc0-bd63-0367fa8fa88e admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:03.111631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d34973-24e8-4b4f-9a92-cba9d73e4b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:03.112326 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d34973-24e8-4b4f-9a92-cba9d73e4b8c admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:03.112608 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d34973-24e8-4b4f-9a92-cba9d73e4b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:03.112886 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d34973-24e8-4b4f-9a92-cba9d73e4b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:03.113631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d34973-24e8-4b4f-9a92-cba9d73e4b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:03.115434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d34973-24e8-4b4f-9a92-cba9d73e4b8c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:03.127309 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1316/2639] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:03.131638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60df86-261e-4f68-b7c9-ae5b9713dab7 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:03.142354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60df86-261e-4f68-b7c9-ae5b9713dab7 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:03.146872 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2de244-3ba7-4586-b2fd-c7e939cacc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:03.147758 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2de244-3ba7-4586-b2fd-c7e939cacc9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:03.148066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2de244-3ba7-4586-b2fd-c7e939cacc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:03.148349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2de244-3ba7-4586-b2fd-c7e939cacc9b admin admin] PATH_INFO: `/v3/projects/1a47c8bcc0aa4b349f015b05447de7db/users/8df6187dc8ff4357b2de0577af99c7a3/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:03.175406 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2de244-3ba7-4586-b2fd-c7e939cacc9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a47c8bcc0aa4b349f015b05447de7db, user_id=8df6187dc8ff4357b2de0577af99c7a3, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:03.178041 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2de244-3ba7-4586-b2fd-c7e939cacc9b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:03.192102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1324/2640] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:03 2022] PUT /identity/v3/projects/1a47c8bcc0aa4b349f015b05447de7db/users/8df6187dc8ff4357b2de0577af99c7a3/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:03.196210 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d34973-24e8-4b4f-9a92-cba9d73e4b8c admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:03.207144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d34973-24e8-4b4f-9a92-cba9d73e4b8c admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:03.266523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1354e-8cf9-4e98-bd7f-136072afe86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:03.267235 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1354e-8cf9-4e98-bd7f-136072afe86f admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:03.267522 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1354e-8cf9-4e98-bd7f-136072afe86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:03.267804 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1354e-8cf9-4e98-bd7f-136072afe86f admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:03.268537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1354e-8cf9-4e98-bd7f-136072afe86f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:03.270135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1354e-8cf9-4e98-bd7f-136072afe86f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:03.279986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1317/2641] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:03.284586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2de244-3ba7-4586-b2fd-c7e939cacc9b admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:03.294818 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2de244-3ba7-4586-b2fd-c7e939cacc9b admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:03.300283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733189f5-00c1-4b75-bff4-7e38f8165631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:03.301264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733189f5-00c1-4b75-bff4-7e38f8165631 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:03.301641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733189f5-00c1-4b75-bff4-7e38f8165631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:03.301913 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733189f5-00c1-4b75-bff4-7e38f8165631 admin admin] PATH_INFO: `/v3/projects/1a47c8bcc0aa4b349f015b05447de7db/users/8df6187dc8ff4357b2de0577af99c7a3/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:03.304287 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733189f5-00c1-4b75-bff4-7e38f8165631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a47c8bcc0aa4b349f015b05447de7db, user_id=8df6187dc8ff4357b2de0577af99c7a3, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:03.306986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733189f5-00c1-4b75-bff4-7e38f8165631 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:03.318467 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1325/2642] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:03 2022] PUT /identity/v3/projects/1a47c8bcc0aa4b349f015b05447de7db/users/8df6187dc8ff4357b2de0577af99c7a3/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:03.328802 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66829b2-52d3-41c2-b188-de0e6e952bab None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:03.329247 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66829b2-52d3-41c2-b188-de0e6e952bab None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:03.329538 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66829b2-52d3-41c2-b188-de0e6e952bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:03.358145 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-d66829b2-52d3-41c2-b188-de0e6e952bab None None] MFA Rules not processed for user `8df6187dc8ff4357b2de0577af99c7a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:03.360025 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d66829b2-52d3-41c2-b188-de0e6e952bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:03.433420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1318/2643] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:03 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:03.445023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733189f5-00c1-4b75-bff4-7e38f8165631 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:03.446915 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-733189f5-00c1-4b75-bff4-7e38f8165631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:03.507249 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733189f5-00c1-4b75-bff4-7e38f8165631 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:03.515446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0b195-7cf3-427b-958d-4777880892a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:03.516181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0b195-7cf3-427b-958d-4777880892a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:03.516465 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0b195-7cf3-427b-958d-4777880892a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:03.516741 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0b195-7cf3-427b-958d-4777880892a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:03.532146 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0b195-7cf3-427b-958d-4777880892a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:03.533578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0b195-7cf3-427b-958d-4777880892a4 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:03.543742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1326/2644] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:03.922316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66829b2-52d3-41c2-b188-de0e6e952bab None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:03.928001 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:03.993161 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66829b2-52d3-41c2-b188-de0e6e952bab None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:03.998094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7485bb-3629-4544-9401-5a9b73935da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:03.999242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7485bb-3629-4544-9401-5a9b73935da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:03.999533 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7485bb-3629-4544-9401-5a9b73935da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:03.999810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7485bb-3629-4544-9401-5a9b73935da2 admin admin] PATH_INFO: `/v3/users/9dd4d112ea504fc991f4347eceb72183` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:04.001281 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7485bb-3629-4544-9401-5a9b73935da2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dd4d112ea504fc991f4347eceb72183)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:04.003242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7485bb-3629-4544-9401-5a9b73935da2 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:04.059142 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-db7485bb-3629-4544-9401-5a9b73935da2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dd4d112ea504fc991f4347eceb72183'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:04.071125 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-db7485bb-3629-4544-9401-5a9b73935da2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dd4d112ea504fc991f4347eceb72183'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:04.078866 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-db7485bb-3629-4544-9401-5a9b73935da2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dd4d112ea504fc991f4347eceb72183'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:04.079563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-db7485bb-3629-4544-9401-5a9b73935da2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dd4d112ea504fc991f4347eceb72183'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:04.089233 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-db7485bb-3629-4544-9401-5a9b73935da2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dd4d112ea504fc991f4347eceb72183'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:04.096450 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1319/2645] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:20:03 2022] DELETE /identity/v3/users/9dd4d112ea504fc991f4347eceb72183 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:04.100930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e138336-3cca-4f49-8267-a91cdaecc65d None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:04.101412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e138336-3cca-4f49-8267-a91cdaecc65d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:04.101512 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e138336-3cca-4f49-8267-a91cdaecc65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:04.118597 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-9e138336-3cca-4f49-8267-a91cdaecc65d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:04.119808 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9e138336-3cca-4f49-8267-a91cdaecc65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:04.182066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1327/2646] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:04.192417 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7485bb-3629-4544-9401-5a9b73935da2 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:04.193867 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-db7485bb-3629-4544-9401-5a9b73935da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:04.253197 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7485bb-3629-4544-9401-5a9b73935da2 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:04.256898 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dacc3c-a2c3-4eb3-a088-bf400b0c3c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:04.257551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dacc3c-a2c3-4eb3-a088-bf400b0c3c5f service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:04.257837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dacc3c-a2c3-4eb3-a088-bf400b0c3c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:04.258118 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dacc3c-a2c3-4eb3-a088-bf400b0c3c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:04.268102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dacc3c-a2c3-4eb3-a088-bf400b0c3c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:04.269575 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dacc3c-a2c3-4eb3-a088-bf400b0c3c5f service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:04.278908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1320/2647] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:04.566499 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e138336-3cca-4f49-8267-a91cdaecc65d None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:04.568036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9e138336-3cca-4f49-8267-a91cdaecc65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:04.580807 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e138336-3cca-4f49-8267-a91cdaecc65d None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:04.585682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a8dd5-a4fd-4bb1-bd90-a79db935075a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:04.586638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a8dd5-a4fd-4bb1-bd90-a79db935075a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:04.586967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a8dd5-a4fd-4bb1-bd90-a79db935075a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:04.587251 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a8dd5-a4fd-4bb1-bd90-a79db935075a admin admin] PATH_INFO: `/v3/projects/49dbfbad631a4c719daf89c42a7246bc` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:04.589187 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a8dd5-a4fd-4bb1-bd90-a79db935075a admin admin] RBAC: Authorizing `identity:delete_project(project_id=49dbfbad631a4c719daf89c42a7246bc)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:04.591195 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a8dd5-a4fd-4bb1-bd90-a79db935075a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:04.606049 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:04.643782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-937a8dd5-a4fd-4bb1-bd90-a79db935075a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49dbfbad631a4c719daf89c42a7246bc', 'request_id': 'req-937a8dd5-a4fd-4bb1-bd90-a79db935075a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:04.653006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-937a8dd5-a4fd-4bb1-bd90-a79db935075a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49dbfbad631a4c719daf89c42a7246bc', 'request_id': 'req-937a8dd5-a4fd-4bb1-bd90-a79db935075a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:04.660073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-937a8dd5-a4fd-4bb1-bd90-a79db935075a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49dbfbad631a4c719daf89c42a7246bc', 'request_id': 'req-937a8dd5-a4fd-4bb1-bd90-a79db935075a'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:04.673236 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-937a8dd5-a4fd-4bb1-bd90-a79db935075a admin admin] The token cache is being invalidate because project 49dbfbad631a4c719daf89c42a7246bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:04.673527 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-937a8dd5-a4fd-4bb1-bd90-a79db935075a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:04.674145 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-937a8dd5-a4fd-4bb1-bd90-a79db935075a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:04.675707 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1328/2648] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:04 2022] DELETE /identity/v3/projects/49dbfbad631a4c719daf89c42a7246bc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:04.679703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dacc3c-a2c3-4eb3-a088-bf400b0c3c5f service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:04.690047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dacc3c-a2c3-4eb3-a088-bf400b0c3c5f service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:04.714957 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:04.750011 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f2d11-b175-4f9d-aeda-f07035c31a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:04.751086 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f2d11-b175-4f9d-aeda-f07035c31a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:04.751373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f2d11-b175-4f9d-aeda-f07035c31a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:04.751652 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f2d11-b175-4f9d-aeda-f07035c31a9a admin admin] PATH_INFO: `/v3/users/8df6187dc8ff4357b2de0577af99c7a3` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:04.752851 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f2d11-b175-4f9d-aeda-f07035c31a9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df6187dc8ff4357b2de0577af99c7a3)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:04.754814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5f2d11-b175-4f9d-aeda-f07035c31a9a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:04.798737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4c5f2d11-b175-4f9d-aeda-f07035c31a9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df6187dc8ff4357b2de0577af99c7a3'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:04.810294 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4c5f2d11-b175-4f9d-aeda-f07035c31a9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df6187dc8ff4357b2de0577af99c7a3'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:04.821910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4c5f2d11-b175-4f9d-aeda-f07035c31a9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df6187dc8ff4357b2de0577af99c7a3'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:04.822814 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4c5f2d11-b175-4f9d-aeda-f07035c31a9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df6187dc8ff4357b2de0577af99c7a3'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:04.831809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-4c5f2d11-b175-4f9d-aeda-f07035c31a9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df6187dc8ff4357b2de0577af99c7a3'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:04.837876 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1321/2649] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:20:04 2022] DELETE /identity/v3/users/8df6187dc8ff4357b2de0577af99c7a3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:05.212117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a8dd5-a4fd-4bb1-bd90-a79db935075a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:05.214227 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-937a8dd5-a4fd-4bb1-bd90-a79db935075a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:05.228635 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a8dd5-a4fd-4bb1-bd90-a79db935075a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:05.293215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069ee76-eb85-4817-9833-68e813f99187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:05.294085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069ee76-eb85-4817-9833-68e813f99187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:05.294372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069ee76-eb85-4817-9833-68e813f99187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:05.294651 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069ee76-eb85-4817-9833-68e813f99187 admin admin] PATH_INFO: `/v3/projects/1a47c8bcc0aa4b349f015b05447de7db` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:05.295868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069ee76-eb85-4817-9833-68e813f99187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a47c8bcc0aa4b349f015b05447de7db)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:05.297879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069ee76-eb85-4817-9833-68e813f99187 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:05.347511 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-4069ee76-eb85-4817-9833-68e813f99187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a47c8bcc0aa4b349f015b05447de7db', 'request_id': 'req-4069ee76-eb85-4817-9833-68e813f99187'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:05.353271 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-4069ee76-eb85-4817-9833-68e813f99187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a47c8bcc0aa4b349f015b05447de7db', 'request_id': 'req-4069ee76-eb85-4817-9833-68e813f99187'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:05.360198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-4069ee76-eb85-4817-9833-68e813f99187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a47c8bcc0aa4b349f015b05447de7db', 'request_id': 'req-4069ee76-eb85-4817-9833-68e813f99187'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:05.372129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-4069ee76-eb85-4817-9833-68e813f99187 admin admin] The token cache is being invalidate because project 1a47c8bcc0aa4b349f015b05447de7db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:05.372424 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-4069ee76-eb85-4817-9833-68e813f99187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:05.373124 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-4069ee76-eb85-4817-9833-68e813f99187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:05.374920 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1329/2650] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:05 2022] DELETE /identity/v3/projects/1a47c8bcc0aa4b349f015b05447de7db => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:05.380703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee3af7-d468-44d8-a1bd-cdb850d36592 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:05.381302 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee3af7-d468-44d8-a1bd-cdb850d36592 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:05.381596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee3af7-d468-44d8-a1bd-cdb850d36592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:05.400198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-04ee3af7-d468-44d8-a1bd-cdb850d36592 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:05.401403 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-04ee3af7-d468-44d8-a1bd-cdb850d36592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:05.462595 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1322/2651] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:05.471922 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec63360-5fe3-434c-9932-d5f9e6815db4 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:05.471922 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec63360-5fe3-434c-9932-d5f9e6815db4 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:05.471922 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec63360-5fe3-434c-9932-d5f9e6815db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:05.491120 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-0ec63360-5fe3-434c-9932-d5f9e6815db4 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:05.492354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-0ec63360-5fe3-434c-9932-d5f9e6815db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:05.500873 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1330/2652] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:05.505177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee3af7-d468-44d8-a1bd-cdb850d36592 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:05.515038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee3af7-d468-44d8-a1bd-cdb850d36592 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:05.525232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966982d6-d975-445f-ba81-c5a64f6dcf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:05.525232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966982d6-d975-445f-ba81-c5a64f6dcf4b admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:05.525232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966982d6-d975-445f-ba81-c5a64f6dcf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:05.525232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966982d6-d975-445f-ba81-c5a64f6dcf4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:05.526079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966982d6-d975-445f-ba81-c5a64f6dcf4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:20:05.526079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966982d6-d975-445f-ba81-c5a64f6dcf4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:05.526079 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966982d6-d975-445f-ba81-c5a64f6dcf4b admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:05.538850 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1323/2653] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:20:05.542578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec63360-5fe3-434c-9932-d5f9e6815db4 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:05.556121 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec63360-5fe3-434c-9932-d5f9e6815db4 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:05.560750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12287691-4234-413d-8ebb-8b656b55cd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:05.561413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12287691-4234-413d-8ebb-8b656b55cd1c admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:05.562295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12287691-4234-413d-8ebb-8b656b55cd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:05.562580 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12287691-4234-413d-8ebb-8b656b55cd1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:05.563311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12287691-4234-413d-8ebb-8b656b55cd1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:05.565287 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12287691-4234-413d-8ebb-8b656b55cd1c admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:05.595404 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1331/2654] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:20:05 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:05.599640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966982d6-d975-445f-ba81-c5a64f6dcf4b admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:05.610091 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966982d6-d975-445f-ba81-c5a64f6dcf4b admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:05.642635 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:05.672098 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da6f0d-e39d-408d-b73b-4956e428e004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:05.672765 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da6f0d-e39d-408d-b73b-4956e428e004 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:05.673050 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da6f0d-e39d-408d-b73b-4956e428e004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:05.673325 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da6f0d-e39d-408d-b73b-4956e428e004 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:05.674017 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da6f0d-e39d-408d-b73b-4956e428e004 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:05.675940 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da6f0d-e39d-408d-b73b-4956e428e004 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:05.698565 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1324/2655] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:20:05 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:05.703133 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12287691-4234-413d-8ebb-8b656b55cd1c admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:05.708586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:05.713723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12287691-4234-413d-8ebb-8b656b55cd1c admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:05.720211 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1e4a2-0a3d-4651-bbc3-8aa786e46bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:05.721118 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1e4a2-0a3d-4651-bbc3-8aa786e46bcb admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:05.721580 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1e4a2-0a3d-4651-bbc3-8aa786e46bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:05.722052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1e4a2-0a3d-4651-bbc3-8aa786e46bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:05.722847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1e4a2-0a3d-4651-bbc3-8aa786e46bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:05.724484 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd1e4a2-0a3d-4651-bbc3-8aa786e46bcb admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:05.735804 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1332/2656] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:05.739756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da6f0d-e39d-408d-b73b-4956e428e004 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:05.750081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da6f0d-e39d-408d-b73b-4956e428e004 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:05.755176 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2071fdc-44b7-499f-941b-d776206ab754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:05.755859 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2071fdc-44b7-499f-941b-d776206ab754 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:05.756141 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2071fdc-44b7-499f-941b-d776206ab754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:05.756415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2071fdc-44b7-499f-941b-d776206ab754 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:05.757151 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2071fdc-44b7-499f-941b-d776206ab754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:05.758787 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2071fdc-44b7-499f-941b-d776206ab754 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:05.769472 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1325/2657] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:05.773602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1e4a2-0a3d-4651-bbc3-8aa786e46bcb admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:05.784259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1e4a2-0a3d-4651-bbc3-8aa786e46bcb admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:05.788807 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf162fa-eed5-4429-8cbc-775360e5f6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:05.789703 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf162fa-eed5-4429-8cbc-775360e5f6ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:05.789985 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf162fa-eed5-4429-8cbc-775360e5f6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:05.790262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf162fa-eed5-4429-8cbc-775360e5f6ce admin admin] PATH_INFO: `/v3/projects/f527b5fdd9f846f8a58d744d396dd095/users/43b26c04e7c445c49ddf25ad74e19524/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:05.806039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf162fa-eed5-4429-8cbc-775360e5f6ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=f527b5fdd9f846f8a58d744d396dd095, user_id=43b26c04e7c445c49ddf25ad74e19524, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:05.808718 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf162fa-eed5-4429-8cbc-775360e5f6ce admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:05.821639 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1333/2658] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:05 2022] PUT /identity/v3/projects/f527b5fdd9f846f8a58d744d396dd095/users/43b26c04e7c445c49ddf25ad74e19524/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:05.832975 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab04008-ea2a-49d5-82fd-26436882c1ae None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:05.833259 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab04008-ea2a-49d5-82fd-26436882c1ae None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:05.833533 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab04008-ea2a-49d5-82fd-26436882c1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:05.862856 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-0ab04008-ea2a-49d5-82fd-26436882c1ae None None] MFA Rules not processed for user `43b26c04e7c445c49ddf25ad74e19524`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:05.864124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-0ab04008-ea2a-49d5-82fd-26436882c1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:05.939104 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1326/2659] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:05.944365 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf162fa-eed5-4429-8cbc-775360e5f6ce admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:06.033802 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf162fa-eed5-4429-8cbc-775360e5f6ce admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:06.039270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283837a-6f01-4c8e-a1ca-3163c945c5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:06.039938 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283837a-6f01-4c8e-a1ca-3163c945c5a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:06.040220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283837a-6f01-4c8e-a1ca-3163c945c5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:06.040872 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283837a-6f01-4c8e-a1ca-3163c945c5a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:06.041206 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283837a-6f01-4c8e-a1ca-3163c945c5a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:06.043274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283837a-6f01-4c8e-a1ca-3163c945c5a2 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:06.082387 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1334/2660] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:20:05 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:20:06.083600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab04008-ea2a-49d5-82fd-26436882c1ae None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:06.096256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab04008-ea2a-49d5-82fd-26436882c1ae None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:06.163761 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fea52-5d17-40af-826a-7327e9e837e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:06.164527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fea52-5d17-40af-826a-7327e9e837e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:06.164831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fea52-5d17-40af-826a-7327e9e837e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:06.165135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fea52-5d17-40af-826a-7327e9e837e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:06.165837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fea52-5d17-40af-826a-7327e9e837e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:06.167876 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fea52-5d17-40af-826a-7327e9e837e2 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:06.190008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1327/2661] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:20:06 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:06.194031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283837a-6f01-4c8e-a1ca-3163c945c5a2 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:06.205345 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283837a-6f01-4c8e-a1ca-3163c945c5a2 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:06.210012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96923871-b14a-44d7-87b3-67959d2420a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:06.210691 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96923871-b14a-44d7-87b3-67959d2420a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:06.211006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96923871-b14a-44d7-87b3-67959d2420a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:06.211272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96923871-b14a-44d7-87b3-67959d2420a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:06.212002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96923871-b14a-44d7-87b3-67959d2420a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:06.213693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96923871-b14a-44d7-87b3-67959d2420a8 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:06.224890 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1335/2662] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:06.229042 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fea52-5d17-40af-826a-7327e9e837e2 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:06.241018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fea52-5d17-40af-826a-7327e9e837e2 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:06.245726 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8308f5-d273-4fa6-b1b5-7c2f6d1c74cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:06.246606 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8308f5-d273-4fa6-b1b5-7c2f6d1c74cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:06.246920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8308f5-d273-4fa6-b1b5-7c2f6d1c74cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:06.247199 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8308f5-d273-4fa6-b1b5-7c2f6d1c74cc admin admin] PATH_INFO: `/v3/projects/f5f888c9eddf4809bcd835ae68eadeaf/users/ec0a1589e07d4e17a449943e7b17c506/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:06.265901 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8308f5-d273-4fa6-b1b5-7c2f6d1c74cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f888c9eddf4809bcd835ae68eadeaf, user_id=ec0a1589e07d4e17a449943e7b17c506, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:06.268669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8308f5-d273-4fa6-b1b5-7c2f6d1c74cc admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:06.282293 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1328/2663] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:06 2022] PUT /identity/v3/projects/f5f888c9eddf4809bcd835ae68eadeaf/users/ec0a1589e07d4e17a449943e7b17c506/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:06.286414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96923871-b14a-44d7-87b3-67959d2420a8 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:06.298154 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96923871-b14a-44d7-87b3-67959d2420a8 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:06.379905 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760608f-ae05-4ecf-b209-ef9c9aaa81af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:06.380593 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760608f-ae05-4ecf-b209-ef9c9aaa81af admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:06.381013 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760608f-ae05-4ecf-b209-ef9c9aaa81af admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:06.381306 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760608f-ae05-4ecf-b209-ef9c9aaa81af admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:06.382236 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760608f-ae05-4ecf-b209-ef9c9aaa81af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:06.384372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760608f-ae05-4ecf-b209-ef9c9aaa81af admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:06.397573 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1336/2664] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:06.398926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8308f5-d273-4fa6-b1b5-7c2f6d1c74cc admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:06.410492 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8308f5-d273-4fa6-b1b5-7c2f6d1c74cc admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:06.425988 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6fa77-c02f-4c76-b483-7874a484c1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:06.426943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6fa77-c02f-4c76-b483-7874a484c1f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:06.427253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6fa77-c02f-4c76-b483-7874a484c1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:06.427529 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6fa77-c02f-4c76-b483-7874a484c1f9 admin admin] PATH_INFO: `/v3/projects/f5f888c9eddf4809bcd835ae68eadeaf/users/ec0a1589e07d4e17a449943e7b17c506/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:06.429658 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6fa77-c02f-4c76-b483-7874a484c1f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f888c9eddf4809bcd835ae68eadeaf, user_id=ec0a1589e07d4e17a449943e7b17c506, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:06.432332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6fa77-c02f-4c76-b483-7874a484c1f9 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:06.434946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:06.451831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1329/2665] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:06 2022] PUT /identity/v3/projects/f5f888c9eddf4809bcd835ae68eadeaf/users/ec0a1589e07d4e17a449943e7b17c506/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:06.463954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02109bf3-c650-4fff-a702-a53651730df4 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:06.464321 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02109bf3-c650-4fff-a702-a53651730df4 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:06.464697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02109bf3-c650-4fff-a702-a53651730df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:06.495627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-02109bf3-c650-4fff-a702-a53651730df4 None None] MFA Rules not processed for user `ec0a1589e07d4e17a449943e7b17c506`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:06.496750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-02109bf3-c650-4fff-a702-a53651730df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:06.530283 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:06.584256 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1337/2666] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:06.595002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6fa77-c02f-4c76-b483-7874a484c1f9 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:06.596755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8ae6fa77-c02f-4c76-b483-7874a484c1f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:06.659650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6fa77-c02f-4c76-b483-7874a484c1f9 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:06.663275 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca43d5-c101-4bc4-8da1-b2f0b650acf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:06.663932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca43d5-c101-4bc4-8da1-b2f0b650acf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:06.664220 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca43d5-c101-4bc4-8da1-b2f0b650acf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:06.664504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca43d5-c101-4bc4-8da1-b2f0b650acf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:06.725014 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca43d5-c101-4bc4-8da1-b2f0b650acf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:06.726502 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca43d5-c101-4bc4-8da1-b2f0b650acf1 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:06.735389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1330/2667] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:12.513049 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02109bf3-c650-4fff-a702-a53651730df4 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:12.515205 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-02109bf3-c650-4fff-a702-a53651730df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:12.575652 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02109bf3-c650-4fff-a702-a53651730df4 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:12.580046 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9e742-7a8e-4466-8fa1-4d14ad987a07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:12.580687 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9e742-7a8e-4466-8fa1-4d14ad987a07 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:12.580971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9e742-7a8e-4466-8fa1-4d14ad987a07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:12.581247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a9e742-7a8e-4466-8fa1-4d14ad987a07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:12.583035 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-03a9e742-7a8e-4466-8fa1-4d14ad987a07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:12.647165 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9e742-7a8e-4466-8fa1-4d14ad987a07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:12.648642 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a9e742-7a8e-4466-8fa1-4d14ad987a07 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:12.660287 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1338/2668] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:12 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:13.104345 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca43d5-c101-4bc4-8da1-b2f0b650acf1 service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:13.114064 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca43d5-c101-4bc4-8da1-b2f0b650acf1 service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:13.117950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629e102-f06f-4080-a57d-10baecbf95bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:13.118665 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629e102-f06f-4080-a57d-10baecbf95bc service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:13.119198 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629e102-f06f-4080-a57d-10baecbf95bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:13.119546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629e102-f06f-4080-a57d-10baecbf95bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:13.129676 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629e102-f06f-4080-a57d-10baecbf95bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:13.131320 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629e102-f06f-4080-a57d-10baecbf95bc service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:13.143596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1331/2669] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:22.657437 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=67 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:23.137812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:INSERT=1,keystone:SELECT=56 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:23.259879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9e742-7a8e-4466-8fa1-4d14ad987a07 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:23.324262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a9e742-7a8e-4466-8fa1-4d14ad987a07 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:23.329192 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c79619-c7b4-497f-8bbd-e67629bad01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:23.330061 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c79619-c7b4-497f-8bbd-e67629bad01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:23.330343 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c79619-c7b4-497f-8bbd-e67629bad01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:23.330618 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c79619-c7b4-497f-8bbd-e67629bad01c admin admin] PATH_INFO: `/v3/users/43b26c04e7c445c49ddf25ad74e19524` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:23.331842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c79619-c7b4-497f-8bbd-e67629bad01c admin admin] RBAC: Authorizing `identity:delete_user(user_id=43b26c04e7c445c49ddf25ad74e19524)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:23.333776 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c79619-c7b4-497f-8bbd-e67629bad01c admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:23.380354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-e5c79619-c7b4-497f-8bbd-e67629bad01c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43b26c04e7c445c49ddf25ad74e19524'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:23.393910 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-e5c79619-c7b4-497f-8bbd-e67629bad01c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43b26c04e7c445c49ddf25ad74e19524'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:23.400947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-e5c79619-c7b4-497f-8bbd-e67629bad01c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43b26c04e7c445c49ddf25ad74e19524'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:23.401664 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-e5c79619-c7b4-497f-8bbd-e67629bad01c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43b26c04e7c445c49ddf25ad74e19524'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:23.410895 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-e5c79619-c7b4-497f-8bbd-e67629bad01c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43b26c04e7c445c49ddf25ad74e19524'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:23.418102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1339/2670] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:20:23 2022] DELETE /identity/v3/users/43b26c04e7c445c49ddf25ad74e19524 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:23.422942 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736d028-1642-435b-adef-989d6e2fd4b1 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:23.423596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736d028-1642-435b-adef-989d6e2fd4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:23.423875 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736d028-1642-435b-adef-989d6e2fd4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:23.441770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-b736d028-1642-435b-adef-989d6e2fd4b1 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:23.442968 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b736d028-1642-435b-adef-989d6e2fd4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:23.507295 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1332/2671] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:20:23.517684 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c79619-c7b4-497f-8bbd-e67629bad01c admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:23.519222 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e5c79619-c7b4-497f-8bbd-e67629bad01c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:23.580821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c79619-c7b4-497f-8bbd-e67629bad01c admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:23.584427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae00c8-1765-4294-b881-6d89df27acb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:23.585080 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae00c8-1765-4294-b881-6d89df27acb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:23.585365 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae00c8-1765-4294-b881-6d89df27acb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:23.585646 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae00c8-1765-4294-b881-6d89df27acb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:23.594989 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae00c8-1765-4294-b881-6d89df27acb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:23.596410 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae00c8-1765-4294-b881-6d89df27acb3 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:23.605853 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1340/2672] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:23.932241 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736d028-1642-435b-adef-989d6e2fd4b1 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:23.934034 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-b736d028-1642-435b-adef-989d6e2fd4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:23.959015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736d028-1642-435b-adef-989d6e2fd4b1 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:23.964479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:23.965363 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:23.965645 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:23.965924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71 admin admin] PATH_INFO: `/v3/projects/f527b5fdd9f846f8a58d744d396dd095` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:23.967730 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f527b5fdd9f846f8a58d744d396dd095)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:23.970256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:24.019899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f527b5fdd9f846f8a58d744d396dd095', 'request_id': 'req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.025729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f527b5fdd9f846f8a58d744d396dd095', 'request_id': 'req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.033510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f527b5fdd9f846f8a58d744d396dd095', 'request_id': 'req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.045941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71 admin admin] The token cache is being invalidate because project f527b5fdd9f846f8a58d744d396dd095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:24.046317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.046920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.048459 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1333/2673] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:23 2022] DELETE /identity/v3/projects/f527b5fdd9f846f8a58d744d396dd095 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:24.052579 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae00c8-1765-4294-b881-6d89df27acb3 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:24.063793 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae00c8-1765-4294-b881-6d89df27acb3 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:24.083948 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:24.125369 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1596267-1920-4cfd-a984-1afb81a5f295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:24.126346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1596267-1920-4cfd-a984-1afb81a5f295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:24.126643 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1596267-1920-4cfd-a984-1afb81a5f295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:24.126957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1596267-1920-4cfd-a984-1afb81a5f295 admin admin] PATH_INFO: `/v3/users/ec0a1589e07d4e17a449943e7b17c506` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:24.128198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1596267-1920-4cfd-a984-1afb81a5f295 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec0a1589e07d4e17a449943e7b17c506)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:24.130067 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1596267-1920-4cfd-a984-1afb81a5f295 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:24.182060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-e1596267-1920-4cfd-a984-1afb81a5f295 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec0a1589e07d4e17a449943e7b17c506'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.195025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-e1596267-1920-4cfd-a984-1afb81a5f295 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec0a1589e07d4e17a449943e7b17c506'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.203030 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-e1596267-1920-4cfd-a984-1afb81a5f295 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec0a1589e07d4e17a449943e7b17c506'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.203867 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-e1596267-1920-4cfd-a984-1afb81a5f295 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec0a1589e07d4e17a449943e7b17c506'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.211704 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-e1596267-1920-4cfd-a984-1afb81a5f295 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec0a1589e07d4e17a449943e7b17c506'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.218595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1341/2674] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:20:24 2022] DELETE /identity/v3/users/ec0a1589e07d4e17a449943e7b17c506 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:24.497918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:24.500169 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:24.513393 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7961b5-3755-4150-ae6d-6ff2d8a5ae71 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:24.572604 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462d859-52b3-4b48-ba20-5764106908f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:24.573474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462d859-52b3-4b48-ba20-5764106908f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:24.573760 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462d859-52b3-4b48-ba20-5764106908f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:24.574040 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462d859-52b3-4b48-ba20-5764106908f1 admin admin] PATH_INFO: `/v3/projects/f5f888c9eddf4809bcd835ae68eadeaf` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:24.575750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462d859-52b3-4b48-ba20-5764106908f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5f888c9eddf4809bcd835ae68eadeaf)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:24.578290 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462d859-52b3-4b48-ba20-5764106908f1 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:24.607584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:24.625899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1462d859-52b3-4b48-ba20-5764106908f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5f888c9eddf4809bcd835ae68eadeaf', 'request_id': 'req-1462d859-52b3-4b48-ba20-5764106908f1'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.631436 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1462d859-52b3-4b48-ba20-5764106908f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5f888c9eddf4809bcd835ae68eadeaf', 'request_id': 'req-1462d859-52b3-4b48-ba20-5764106908f1'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.638605 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1462d859-52b3-4b48-ba20-5764106908f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5f888c9eddf4809bcd835ae68eadeaf', 'request_id': 'req-1462d859-52b3-4b48-ba20-5764106908f1'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.650504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1462d859-52b3-4b48-ba20-5764106908f1 admin admin] The token cache is being invalidate because project f5f888c9eddf4809bcd835ae68eadeaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:24.650831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1462d859-52b3-4b48-ba20-5764106908f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.651468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-1462d859-52b3-4b48-ba20-5764106908f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.652995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1334/2675] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:24 2022] DELETE /identity/v3/projects/f5f888c9eddf4809bcd835ae68eadeaf => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:24.659106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6dc7c-a7d6-4119-9e2b-7027bd74cb39 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:24.659401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6dc7c-a7d6-4119-9e2b-7027bd74cb39 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:24.659752 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6dc7c-a7d6-4119-9e2b-7027bd74cb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:24.677246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-57b6dc7c-a7d6-4119-9e2b-7027bd74cb39 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:24.678393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-57b6dc7c-a7d6-4119-9e2b-7027bd74cb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:24.737343 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1342/2676] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:24.745843 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bbf7d-614b-47b3-ab27-e708c8f9bf09 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:24.746135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bbf7d-614b-47b3-ab27-e708c8f9bf09 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:24.746419 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bbf7d-614b-47b3-ab27-e708c8f9bf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:24.765379 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-e28bbf7d-614b-47b3-ab27-e708c8f9bf09 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:24.766586 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e28bbf7d-614b-47b3-ab27-e708c8f9bf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:24.774597 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1335/2677] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:24.778821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6dc7c-a7d6-4119-9e2b-7027bd74cb39 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:24.789425 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6dc7c-a7d6-4119-9e2b-7027bd74cb39 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:24.794209 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927eea06-fc4a-4ee7-b7c9-5479d29ff63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:24.794900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927eea06-fc4a-4ee7-b7c9-5479d29ff63f admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:24.795190 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927eea06-fc4a-4ee7-b7c9-5479d29ff63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:24.796167 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927eea06-fc4a-4ee7-b7c9-5479d29ff63f admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:24.797183 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927eea06-fc4a-4ee7-b7c9-5479d29ff63f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:20:24.797502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927eea06-fc4a-4ee7-b7c9-5479d29ff63f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:24.799150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927eea06-fc4a-4ee7-b7c9-5479d29ff63f admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:24.811372 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28bbf7d-614b-47b3-ab27-e708c8f9bf09 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:24.815024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1343/2678] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:20:24.815170 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e28bbf7d-614b-47b3-ab27-e708c8f9bf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:24.815884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927eea06-fc4a-4ee7-b7c9-5479d29ff63f admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:24.825674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28bbf7d-614b-47b3-ab27-e708c8f9bf09 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:24.825850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927eea06-fc4a-4ee7-b7c9-5479d29ff63f admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:24.830422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4bf79-dafa-45e4-851a-a0c9aaf9fd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:24.830638 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33070f5-7eb6-4b0c-bf7c-be0d3a50aa11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:24.831110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4bf79-dafa-45e4-851a-a0c9aaf9fd7e admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:24.831398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4bf79-dafa-45e4-851a-a0c9aaf9fd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:24.831525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33070f5-7eb6-4b0c-bf7c-be0d3a50aa11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:24.831680 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4bf79-dafa-45e4-851a-a0c9aaf9fd7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:24.831800 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33070f5-7eb6-4b0c-bf7c-be0d3a50aa11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:24.832067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33070f5-7eb6-4b0c-bf7c-be0d3a50aa11 admin admin] PATH_INFO: `/v3/users/a7fa96d72da44c4bad9713076d7c96fa` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:24.832369 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4bf79-dafa-45e4-851a-a0c9aaf9fd7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:24.833249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33070f5-7eb6-4b0c-bf7c-be0d3a50aa11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7fa96d72da44c4bad9713076d7c96fa)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:24.834244 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4bf79-dafa-45e4-851a-a0c9aaf9fd7e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:24.835202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33070f5-7eb6-4b0c-bf7c-be0d3a50aa11 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:24.866606 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1344/2679] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:20:24 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:20:24.868161 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4bf79-dafa-45e4-851a-a0c9aaf9fd7e admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:24.877288 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:24.884622 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b33070f5-7eb6-4b0c-bf7c-be0d3a50aa11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7fa96d72da44c4bad9713076d7c96fa'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.898291 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b33070f5-7eb6-4b0c-bf7c-be0d3a50aa11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7fa96d72da44c4bad9713076d7c96fa'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.905510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b33070f5-7eb6-4b0c-bf7c-be0d3a50aa11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7fa96d72da44c4bad9713076d7c96fa'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.906231 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b33070f5-7eb6-4b0c-bf7c-be0d3a50aa11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7fa96d72da44c4bad9713076d7c96fa'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.913721 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b33070f5-7eb6-4b0c-bf7c-be0d3a50aa11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7fa96d72da44c4bad9713076d7c96fa'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:24.920801 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1336/2680] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:20:24 2022] DELETE /identity/v3/users/a7fa96d72da44c4bad9713076d7c96fa => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:20:24.925367 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d9537-b668-4bcd-a027-1167540d3a4f None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:24.925688 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d9537-b668-4bcd-a027-1167540d3a4f None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:24.925975 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d9537-b668-4bcd-a027-1167540d3a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:24.935333 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4bf79-dafa-45e4-851a-a0c9aaf9fd7e admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:24.943139 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-3f6d9537-b668-4bcd-a027-1167540d3a4f None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:24.944347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3f6d9537-b668-4bcd-a027-1167540d3a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:24.993877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8331e90-5161-4e2a-82ef-66b312becdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:24.994607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8331e90-5161-4e2a-82ef-66b312becdce admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:24.994928 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8331e90-5161-4e2a-82ef-66b312becdce admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:24.995222 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8331e90-5161-4e2a-82ef-66b312becdce admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:24.996031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8331e90-5161-4e2a-82ef-66b312becdce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:24.998153 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8331e90-5161-4e2a-82ef-66b312becdce admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:25.007774 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1337/2681] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:20:25.020072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d9537-b668-4bcd-a027-1167540d3a4f None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:25.021745 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3f6d9537-b668-4bcd-a027-1167540d3a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:25.026262 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1345/2682] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:20:24 2022] POST /identity/v3/users => generated 285 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:20:25.031004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8331e90-5161-4e2a-82ef-66b312becdce admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:25.032689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d8331e90-5161-4e2a-82ef-66b312becdce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:25.048192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:25.049248 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8331e90-5161-4e2a-82ef-66b312becdce admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:25.054440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef4b48-213c-48c0-a4ea-c46c5aefd949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:25.055185 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef4b48-213c-48c0-a4ea-c46c5aefd949 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.055427 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef4b48-213c-48c0-a4ea-c46c5aefd949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.055717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef4b48-213c-48c0-a4ea-c46c5aefd949 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.056453 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef4b48-213c-48c0-a4ea-c46c5aefd949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:25.058079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef4b48-213c-48c0-a4ea-c46c5aefd949 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:25.069805 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1346/2683] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:20:25.074200 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef4b48-213c-48c0-a4ea-c46c5aefd949 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:25.086322 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef4b48-213c-48c0-a4ea-c46c5aefd949 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:25.091589 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b22ce-0543-4dce-bfff-f2786bfe4ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:25.091764 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d9537-b668-4bcd-a027-1167540d3a4f None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:25.091871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b22ce-0543-4dce-bfff-f2786bfe4ebc admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.092826 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b22ce-0543-4dce-bfff-f2786bfe4ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.093138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b22ce-0543-4dce-bfff-f2786bfe4ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.093877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b22ce-0543-4dce-bfff-f2786bfe4ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:25.095520 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b22ce-0543-4dce-bfff-f2786bfe4ebc admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:25.096053 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ef14a-4e53-4a23-b337-ec87662c6488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:25.096783 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ef14a-4e53-4a23-b337-ec87662c6488 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.097071 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ef14a-4e53-4a23-b337-ec87662c6488 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.097346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ef14a-4e53-4a23-b337-ec87662c6488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.106247 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1347/2684] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:20:25.106709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ef14a-4e53-4a23-b337-ec87662c6488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:25.108257 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ef14a-4e53-4a23-b337-ec87662c6488 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:25.110410 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b22ce-0543-4dce-bfff-f2786bfe4ebc admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:25.117964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1338/2685] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:20:25.120022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b22ce-0543-4dce-bfff-f2786bfe4ebc admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:25.125051 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a1def-5d7e-4003-987d-8d9348198096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:25.126337 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a1def-5d7e-4003-987d-8d9348198096 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.126337 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a1def-5d7e-4003-987d-8d9348198096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.126568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a1def-5d7e-4003-987d-8d9348198096 admin admin] PATH_INFO: `/v3/projects/69dd79d577354169bbe59cd570226346/users/8a5fbb1f23d74b209f3f5cb78f2ad01c/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.142106 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a1def-5d7e-4003-987d-8d9348198096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69dd79d577354169bbe59cd570226346, user_id=8a5fbb1f23d74b209f3f5cb78f2ad01c, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:25.144869 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a1def-5d7e-4003-987d-8d9348198096 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:25.158925 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1348/2686] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:25 2022] PUT /identity/v3/projects/69dd79d577354169bbe59cd570226346/users/8a5fbb1f23d74b209f3f5cb78f2ad01c/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:25.171634 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f2ca3-8cad-4a91-acb4-7ebbdaffa607 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.171920 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f2ca3-8cad-4a91-acb4-7ebbdaffa607 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.172200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f2ca3-8cad-4a91-acb4-7ebbdaffa607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.202423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-876f2ca3-8cad-4a91-acb4-7ebbdaffa607 None None] MFA Rules not processed for user `8a5fbb1f23d74b209f3f5cb78f2ad01c`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:25.203881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-876f2ca3-8cad-4a91-acb4-7ebbdaffa607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:25.280859 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1339/2687] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:25.292072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a1def-5d7e-4003-987d-8d9348198096 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:25.294094 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c65a1def-5d7e-4003-987d-8d9348198096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:25.366540 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a1def-5d7e-4003-987d-8d9348198096 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:25.370456 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb971a-b4dc-4b13-96e9-25499109c196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:25.371300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb971a-b4dc-4b13-96e9-25499109c196 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.371665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb971a-b4dc-4b13-96e9-25499109c196 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.372020 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb971a-b4dc-4b13-96e9-25499109c196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.382969 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb971a-b4dc-4b13-96e9-25499109c196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:25.385345 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb971a-b4dc-4b13-96e9-25499109c196 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:25.391650 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:25.397344 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1349/2688] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:25.417194 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb971a-b4dc-4b13-96e9-25499109c196 service cinder] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:25.417415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f2ca3-8cad-4a91-acb4-7ebbdaffa607 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:25.418863 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-876f2ca3-8cad-4a91-acb4-7ebbdaffa607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:25.492303 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb971a-b4dc-4b13-96e9-25499109c196 service cinder] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:25.497775 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d8a69-7bee-4b7f-b686-6964a9616073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:25.499481 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d8a69-7bee-4b7f-b686-6964a9616073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.499858 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d8a69-7bee-4b7f-b686-6964a9616073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.500158 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d8a69-7bee-4b7f-b686-6964a9616073 admin admin] PATH_INFO: `/v3/users/8a5fbb1f23d74b209f3f5cb78f2ad01c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.501476 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d8a69-7bee-4b7f-b686-6964a9616073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a5fbb1f23d74b209f3f5cb78f2ad01c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:25.503458 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d8a69-7bee-4b7f-b686-6964a9616073 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:25.507292 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f2ca3-8cad-4a91-acb4-7ebbdaffa607 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:25.511895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f90bfd-6dd5-4c2d-a3f7-602d945901e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:25.512762 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f90bfd-6dd5-4c2d-a3f7-602d945901e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.513046 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f90bfd-6dd5-4c2d-a3f7-602d945901e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.513332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f90bfd-6dd5-4c2d-a3f7-602d945901e6 admin admin] PATH_INFO: `/v3/projects/93bdfaf479ca496fb687173ef2cb5b5a` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.515579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f90bfd-6dd5-4c2d-a3f7-602d945901e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93bdfaf479ca496fb687173ef2cb5b5a)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:25.518036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f90bfd-6dd5-4c2d-a3f7-602d945901e6 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:25.556623 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-246d8a69-7bee-4b7f-b686-6964a9616073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a5fbb1f23d74b209f3f5cb78f2ad01c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:25.567525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-88f90bfd-6dd5-4c2d-a3f7-602d945901e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93bdfaf479ca496fb687173ef2cb5b5a', 'request_id': 'req-88f90bfd-6dd5-4c2d-a3f7-602d945901e6'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:25.569840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-246d8a69-7bee-4b7f-b686-6964a9616073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a5fbb1f23d74b209f3f5cb78f2ad01c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:25.571791 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:25.574836 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-88f90bfd-6dd5-4c2d-a3f7-602d945901e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93bdfaf479ca496fb687173ef2cb5b5a', 'request_id': 'req-88f90bfd-6dd5-4c2d-a3f7-602d945901e6'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:25.577733 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-246d8a69-7bee-4b7f-b686-6964a9616073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a5fbb1f23d74b209f3f5cb78f2ad01c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:25.578457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-246d8a69-7bee-4b7f-b686-6964a9616073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a5fbb1f23d74b209f3f5cb78f2ad01c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:25.584606 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-88f90bfd-6dd5-4c2d-a3f7-602d945901e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93bdfaf479ca496fb687173ef2cb5b5a', 'request_id': 'req-88f90bfd-6dd5-4c2d-a3f7-602d945901e6'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:25.587015 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-246d8a69-7bee-4b7f-b686-6964a9616073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a5fbb1f23d74b209f3f5cb78f2ad01c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:25.593509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1350/2689] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:20:25 2022] DELETE /identity/v3/users/8a5fbb1f23d74b209f3f5cb78f2ad01c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:25.597393 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-88f90bfd-6dd5-4c2d-a3f7-602d945901e6 admin admin] The token cache is being invalidate because project 93bdfaf479ca496fb687173ef2cb5b5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:25.597701 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-88f90bfd-6dd5-4c2d-a3f7-602d945901e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:25.597858 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586772bd-9e40-4349-983f-abf17a3a8c84 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.598183 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586772bd-9e40-4349-983f-abf17a3a8c84 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.598472 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586772bd-9e40-4349-983f-abf17a3a8c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.598586 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-88f90bfd-6dd5-4c2d-a3f7-602d945901e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:25.600049 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1340/2690] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:25 2022] DELETE /identity/v3/projects/93bdfaf479ca496fb687173ef2cb5b5a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:25.605062 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2c598-de70-47cb-a50e-ad4b88be1ef0 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.605351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2c598-de70-47cb-a50e-ad4b88be1ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.605631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2c598-de70-47cb-a50e-ad4b88be1ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.619539 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-586772bd-9e40-4349-983f-abf17a3a8c84 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:25.620745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-586772bd-9e40-4349-983f-abf17a3a8c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:25.626237 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-3dd2c598-de70-47cb-a50e-ad4b88be1ef0 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:25.627432 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3dd2c598-de70-47cb-a50e-ad4b88be1ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:25.684277 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1351/2691] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:20:25.692426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1341/2692] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:20:25.694004 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692f114-273f-4536-a0d9-0a21e2e9aba3 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.694298 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692f114-273f-4536-a0d9-0a21e2e9aba3 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.694583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692f114-273f-4536-a0d9-0a21e2e9aba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.694792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2c598-de70-47cb-a50e-ad4b88be1ef0 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:25.696434 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3dd2c598-de70-47cb-a50e-ad4b88be1ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:25.714402 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-7692f114-273f-4536-a0d9-0a21e2e9aba3 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:25.715638 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7692f114-273f-4536-a0d9-0a21e2e9aba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:25.725026 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1352/2693] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:20:25.729220 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692f114-273f-4536-a0d9-0a21e2e9aba3 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:25.738346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692f114-273f-4536-a0d9-0a21e2e9aba3 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:25.743104 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc65759-d4cb-44e5-a3c3-0c6d0ce2890b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:25.743687 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc65759-d4cb-44e5-a3c3-0c6d0ce2890b admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.743971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc65759-d4cb-44e5-a3c3-0c6d0ce2890b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.744252 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc65759-d4cb-44e5-a3c3-0c6d0ce2890b admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.744979 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc65759-d4cb-44e5-a3c3-0c6d0ce2890b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:20:25.745289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc65759-d4cb-44e5-a3c3-0c6d0ce2890b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:25.746920 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc65759-d4cb-44e5-a3c3-0c6d0ce2890b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:25.755838 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:25.757840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2c598-de70-47cb-a50e-ad4b88be1ef0 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:25.760999 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1353/2694] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:20:25.762397 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589cdea-8fa0-4eee-9eeb-8a70c333d3e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:25.763191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589cdea-8fa0-4eee-9eeb-8a70c333d3e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.763590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589cdea-8fa0-4eee-9eeb-8a70c333d3e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.764474 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc65759-d4cb-44e5-a3c3-0c6d0ce2890b admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:25.764911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589cdea-8fa0-4eee-9eeb-8a70c333d3e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.774724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589cdea-8fa0-4eee-9eeb-8a70c333d3e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:25.776167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589cdea-8fa0-4eee-9eeb-8a70c333d3e7 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:25.776299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc65759-d4cb-44e5-a3c3-0c6d0ce2890b admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:25.781641 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823569f-0151-4f67-873f-a8c74b0e2579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:25.782289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823569f-0151-4f67-873f-a8c74b0e2579 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.782574 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823569f-0151-4f67-873f-a8c74b0e2579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.782889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823569f-0151-4f67-873f-a8c74b0e2579 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.783577 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823569f-0151-4f67-873f-a8c74b0e2579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:25.785441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823569f-0151-4f67-873f-a8c74b0e2579 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:25.786585 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1342/2695] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:20:25.811096 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1354/2696] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:20:25 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:25.815181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589cdea-8fa0-4eee-9eeb-8a70c333d3e7 service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:25.824571 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589cdea-8fa0-4eee-9eeb-8a70c333d3e7 service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:25.884641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71534e-5c66-46e3-9f37-bbedf879bebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:25.885317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71534e-5c66-46e3-9f37-bbedf879bebd admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.885598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71534e-5c66-46e3-9f37-bbedf879bebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.885873 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71534e-5c66-46e3-9f37-bbedf879bebd admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.886598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71534e-5c66-46e3-9f37-bbedf879bebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:25.888571 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71534e-5c66-46e3-9f37-bbedf879bebd admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:25.911013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1343/2697] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:20:25 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:20:25.915285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823569f-0151-4f67-873f-a8c74b0e2579 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:25.926346 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823569f-0151-4f67-873f-a8c74b0e2579 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:25.931123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a75a6f-7c6a-4ad2-b48f-80207728f90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:25.931923 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a75a6f-7c6a-4ad2-b48f-80207728f90f admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.932311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a75a6f-7c6a-4ad2-b48f-80207728f90f admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.932660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a75a6f-7c6a-4ad2-b48f-80207728f90f admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.933503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a75a6f-7c6a-4ad2-b48f-80207728f90f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:25.935249 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a75a6f-7c6a-4ad2-b48f-80207728f90f admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:25.950949 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1355/2698] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:25.952297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71534e-5c66-46e3-9f37-bbedf879bebd admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:25.962519 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71534e-5c66-46e3-9f37-bbedf879bebd admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:25.968362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a14d62-1c8f-44f0-84d8-b9774de0f650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:25.969324 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a14d62-1c8f-44f0-84d8-b9774de0f650 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:25.969611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a14d62-1c8f-44f0-84d8-b9774de0f650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:25.969884 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a14d62-1c8f-44f0-84d8-b9774de0f650 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:25.970642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a14d62-1c8f-44f0-84d8-b9774de0f650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:25.972335 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a14d62-1c8f-44f0-84d8-b9774de0f650 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:25.982666 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1344/2699] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:20:25.987582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a75a6f-7c6a-4ad2-b48f-80207728f90f admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:25.997584 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a75a6f-7c6a-4ad2-b48f-80207728f90f admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.002930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd003e4-dc2f-4a4d-8ee5-315958c040b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.003887 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd003e4-dc2f-4a4d-8ee5-315958c040b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.004171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd003e4-dc2f-4a4d-8ee5-315958c040b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.004446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd003e4-dc2f-4a4d-8ee5-315958c040b3 admin admin] PATH_INFO: `/v3/projects/6af8a41843f0451e8fe98bb4fa37218a/users/3733fa2e042a4596aef68a01de090293/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.022033 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd003e4-dc2f-4a4d-8ee5-315958c040b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6af8a41843f0451e8fe98bb4fa37218a, user_id=3733fa2e042a4596aef68a01de090293, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.024798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd003e4-dc2f-4a4d-8ee5-315958c040b3 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.037837 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1356/2700] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:25 2022] PUT /identity/v3/projects/6af8a41843f0451e8fe98bb4fa37218a/users/3733fa2e042a4596aef68a01de090293/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:26.048235 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62002d5-e923-4666-9133-48dec62e255f None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.048514 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62002d5-e923-4666-9133-48dec62e255f None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.048799 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62002d5-e923-4666-9133-48dec62e255f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.056164 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:26.073618 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd003e4-dc2f-4a4d-8ee5-315958c040b3 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:26.076060 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-1bd003e4-dc2f-4a4d-8ee5-315958c040b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:26.084654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-c62002d5-e923-4666-9133-48dec62e255f None None] MFA Rules not processed for user `3733fa2e042a4596aef68a01de090293`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:26.085879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c62002d5-e923-4666-9133-48dec62e255f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:26.146725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd003e4-dc2f-4a4d-8ee5-315958c040b3 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.151344 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182dd8d-160b-481e-9b76-0b4bf6c1aecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.152204 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182dd8d-160b-481e-9b76-0b4bf6c1aecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.152486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182dd8d-160b-481e-9b76-0b4bf6c1aecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.152763 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182dd8d-160b-481e-9b76-0b4bf6c1aecd admin admin] PATH_INFO: `/v3/projects/69dd79d577354169bbe59cd570226346` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.153946 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182dd8d-160b-481e-9b76-0b4bf6c1aecd admin admin] RBAC: Authorizing `identity:delete_project(project_id=69dd79d577354169bbe59cd570226346)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.154527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1345/2701] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:26.156674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182dd8d-160b-481e-9b76-0b4bf6c1aecd admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.159889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62002d5-e923-4666-9133-48dec62e255f None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:26.171313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62002d5-e923-4666-9133-48dec62e255f None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.175885 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126757d0-715f-4f55-80a6-edb4be164893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.176537 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126757d0-715f-4f55-80a6-edb4be164893 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.176820 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126757d0-715f-4f55-80a6-edb4be164893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.177095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126757d0-715f-4f55-80a6-edb4be164893 admin admin] PATH_INFO: `/v3/projects` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.177780 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126757d0-715f-4f55-80a6-edb4be164893 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.179704 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126757d0-715f-4f55-80a6-edb4be164893 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.205410 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5182dd8d-160b-481e-9b76-0b4bf6c1aecd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69dd79d577354169bbe59cd570226346', 'request_id': 'req-5182dd8d-160b-481e-9b76-0b4bf6c1aecd'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:26.205850 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1346/2702] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:20:26 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:20:26.209604 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126757d0-715f-4f55-80a6-edb4be164893 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:26.211315 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5182dd8d-160b-481e-9b76-0b4bf6c1aecd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69dd79d577354169bbe59cd570226346', 'request_id': 'req-5182dd8d-160b-481e-9b76-0b4bf6c1aecd'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:26.217977 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5182dd8d-160b-481e-9b76-0b4bf6c1aecd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69dd79d577354169bbe59cd570226346', 'request_id': 'req-5182dd8d-160b-481e-9b76-0b4bf6c1aecd'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:26.230401 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5182dd8d-160b-481e-9b76-0b4bf6c1aecd admin admin] The token cache is being invalidate because project 69dd79d577354169bbe59cd570226346 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:26.230700 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5182dd8d-160b-481e-9b76-0b4bf6c1aecd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:26.231411 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5182dd8d-160b-481e-9b76-0b4bf6c1aecd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:26.234059 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1357/2703] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:26 2022] DELETE /identity/v3/projects/69dd79d577354169bbe59cd570226346 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:20:26.238958 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98fb3c-0276-4e41-a52e-e73d61298ffc None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.239257 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98fb3c-0276-4e41-a52e-e73d61298ffc None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.239545 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98fb3c-0276-4e41-a52e-e73d61298ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.257654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-5f98fb3c-0276-4e41-a52e-e73d61298ffc None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:26.258822 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5f98fb3c-0276-4e41-a52e-e73d61298ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:26.278320 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126757d0-715f-4f55-80a6-edb4be164893 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.285124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb027da2-993b-4d0c-b3a4-ba5f47c9c299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.285910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb027da2-993b-4d0c-b3a4-ba5f47c9c299 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.286313 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb027da2-993b-4d0c-b3a4-ba5f47c9c299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.286650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb027da2-993b-4d0c-b3a4-ba5f47c9c299 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.287421 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb027da2-993b-4d0c-b3a4-ba5f47c9c299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.289181 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:26.289994 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb027da2-993b-4d0c-b3a4-ba5f47c9c299 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.314625 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1347/2704] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:20:26 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:20:26.318721 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb027da2-993b-4d0c-b3a4-ba5f47c9c299 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:26.320290 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-bb027da2-993b-4d0c-b3a4-ba5f47c9c299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:26.322822 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1358/2705] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:20:26.331803 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c3914-eaa6-4a0b-9673-d691b4dbbbda None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.332100 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c3914-eaa6-4a0b-9673-d691b4dbbbda None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.332413 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c3914-eaa6-4a0b-9673-d691b4dbbbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.332600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb027da2-993b-4d0c-b3a4-ba5f47c9c299 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.337176 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb590a-8f45-4bc1-b9d4-bc9fff5c2989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.337826 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb590a-8f45-4bc1-b9d4-bc9fff5c2989 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.338107 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb590a-8f45-4bc1-b9d4-bc9fff5c2989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.338381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb590a-8f45-4bc1-b9d4-bc9fff5c2989 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.339138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb590a-8f45-4bc1-b9d4-bc9fff5c2989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.340710 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb590a-8f45-4bc1-b9d4-bc9fff5c2989 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.350625 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-4a5c3914-eaa6-4a0b-9673-d691b4dbbbda None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:26.354984 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1348/2706] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:26.355161 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-4a5c3914-eaa6-4a0b-9673-d691b4dbbbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:26.355962 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb590a-8f45-4bc1-b9d4-bc9fff5c2989 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:26.360911 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1359/2707] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:26.364833 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c3914-eaa6-4a0b-9673-d691b4dbbbda None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:26.366202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb590a-8f45-4bc1-b9d4-bc9fff5c2989 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.371479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad7455-dd8b-4523-acb1-58108946c35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.372554 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad7455-dd8b-4523-acb1-58108946c35f admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.372833 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad7455-dd8b-4523-acb1-58108946c35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.373105 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad7455-dd8b-4523-acb1-58108946c35f admin admin] PATH_INFO: `/v3/projects/ab02a6020b504783a88d734c329f9330/users/1a0fe805451844ab8e55bb772e1f5f2d/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.374335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c3914-eaa6-4a0b-9673-d691b4dbbbda None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.378962 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2416e2-2812-408a-b0f1-bc0577e2cb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.379620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2416e2-2812-408a-b0f1-bc0577e2cb5b admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.379909 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2416e2-2812-408a-b0f1-bc0577e2cb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.380190 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2416e2-2812-408a-b0f1-bc0577e2cb5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.380916 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2416e2-2812-408a-b0f1-bc0577e2cb5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:20:26.381229 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2416e2-2812-408a-b0f1-bc0577e2cb5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.382840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2416e2-2812-408a-b0f1-bc0577e2cb5b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.391034 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad7455-dd8b-4523-acb1-58108946c35f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab02a6020b504783a88d734c329f9330, user_id=1a0fe805451844ab8e55bb772e1f5f2d, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.393800 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad7455-dd8b-4523-acb1-58108946c35f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.395398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1360/2708] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:20:26.400259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2416e2-2812-408a-b0f1-bc0577e2cb5b admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:26.407776 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1349/2709] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:26 2022] PUT /identity/v3/projects/ab02a6020b504783a88d734c329f9330/users/1a0fe805451844ab8e55bb772e1f5f2d/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:26.411587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad7455-dd8b-4523-acb1-58108946c35f admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:26.414392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2416e2-2812-408a-b0f1-bc0577e2cb5b admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.444567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:26.474526 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475c737-c29c-4a82-8785-36902c8f58d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.475293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475c737-c29c-4a82-8785-36902c8f58d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.475580 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475c737-c29c-4a82-8785-36902c8f58d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.475858 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475c737-c29c-4a82-8785-36902c8f58d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.476548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475c737-c29c-4a82-8785-36902c8f58d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.478583 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475c737-c29c-4a82-8785-36902c8f58d0 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.481055 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad7455-dd8b-4523-acb1-58108946c35f admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.486959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c0430-8348-40c2-8b23-b5fa24a77c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.487635 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c0430-8348-40c2-8b23-b5fa24a77c16 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.487921 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c0430-8348-40c2-8b23-b5fa24a77c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.488203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c0430-8348-40c2-8b23-b5fa24a77c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.488964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c0430-8348-40c2-8b23-b5fa24a77c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.490573 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c0430-8348-40c2-8b23-b5fa24a77c16 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.501923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1350/2710] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:20:26.503031 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c0430-8348-40c2-8b23-b5fa24a77c16 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:26.506190 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-970c0430-8348-40c2-8b23-b5fa24a77c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:26.509482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1361/2711] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:20:26 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:20:26.510290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475c737-c29c-4a82-8785-36902c8f58d0 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:26.519840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c0430-8348-40c2-8b23-b5fa24a77c16 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.573273 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475c737-c29c-4a82-8785-36902c8f58d0 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.579187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12924b86-0054-455e-a8de-6e52f3743e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.579670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41221ea-a132-426a-ae08-b37bb0c93802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.580065 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12924b86-0054-455e-a8de-6e52f3743e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.580349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12924b86-0054-455e-a8de-6e52f3743e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.580575 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41221ea-a132-426a-ae08-b37bb0c93802 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.580701 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12924b86-0054-455e-a8de-6e52f3743e76 admin admin] PATH_INFO: `/v3/users/ff81dd44e702476ea9aab7eea7d82f1c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.580838 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41221ea-a132-426a-ae08-b37bb0c93802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.581148 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41221ea-a132-426a-ae08-b37bb0c93802 admin admin] PATH_INFO: `/v3/projects/ab02a6020b504783a88d734c329f9330/users/1a0fe805451844ab8e55bb772e1f5f2d/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.581816 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12924b86-0054-455e-a8de-6e52f3743e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff81dd44e702476ea9aab7eea7d82f1c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.583189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41221ea-a132-426a-ae08-b37bb0c93802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab02a6020b504783a88d734c329f9330, user_id=1a0fe805451844ab8e55bb772e1f5f2d, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.583780 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12924b86-0054-455e-a8de-6e52f3743e76 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.585846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41221ea-a132-426a-ae08-b37bb0c93802 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.596496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1362/2712] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:26 2022] PUT /identity/v3/projects/ab02a6020b504783a88d734c329f9330/users/1a0fe805451844ab8e55bb772e1f5f2d/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:26.597282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41221ea-a132-426a-ae08-b37bb0c93802 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:26.639950 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:26.646732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-12924b86-0054-455e-a8de-6e52f3743e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff81dd44e702476ea9aab7eea7d82f1c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:26.664435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-12924b86-0054-455e-a8de-6e52f3743e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff81dd44e702476ea9aab7eea7d82f1c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:26.671726 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41221ea-a132-426a-ae08-b37bb0c93802 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.671918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-12924b86-0054-455e-a8de-6e52f3743e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff81dd44e702476ea9aab7eea7d82f1c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:26.672571 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-12924b86-0054-455e-a8de-6e52f3743e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff81dd44e702476ea9aab7eea7d82f1c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:26.676503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b409cb9-e9ef-423a-8472-16ac06e41603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.677184 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b409cb9-e9ef-423a-8472-16ac06e41603 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.677495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b409cb9-e9ef-423a-8472-16ac06e41603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.677747 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b409cb9-e9ef-423a-8472-16ac06e41603 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.678492 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b409cb9-e9ef-423a-8472-16ac06e41603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.680072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-12924b86-0054-455e-a8de-6e52f3743e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff81dd44e702476ea9aab7eea7d82f1c'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:26.680592 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b409cb9-e9ef-423a-8472-16ac06e41603 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.686411 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1351/2713] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:20:26 2022] DELETE /identity/v3/users/ff81dd44e702476ea9aab7eea7d82f1c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:26.688021 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ab7fe-3671-4614-8993-2ee756a2e4f3 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.688306 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ab7fe-3671-4614-8993-2ee756a2e4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.688586 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ab7fe-3671-4614-8993-2ee756a2e4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.706504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1363/2714] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:20:26 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:26.706504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d80644-b393-4d55-b15f-86f0a49feab0 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.706504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d80644-b393-4d55-b15f-86f0a49feab0 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.706504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d80644-b393-4d55-b15f-86f0a49feab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.715409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-d00ab7fe-3671-4614-8993-2ee756a2e4f3 None None] MFA Rules not processed for user `1a0fe805451844ab8e55bb772e1f5f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:26.716642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d00ab7fe-3671-4614-8993-2ee756a2e4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:26.723942 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-c7d80644-b393-4d55-b15f-86f0a49feab0 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:26.725115 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c7d80644-b393-4d55-b15f-86f0a49feab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:26.749420 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:26.788121 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1364/2715] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:26.788883 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d80644-b393-4d55-b15f-86f0a49feab0 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:26.790142 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c7d80644-b393-4d55-b15f-86f0a49feab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:26.796890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1352/2716] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:26.798622 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ab7fe-3671-4614-8993-2ee756a2e4f3 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:26.800126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d00ab7fe-3671-4614-8993-2ee756a2e4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:26.802882 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d80644-b393-4d55-b15f-86f0a49feab0 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.807778 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa2235-9eb4-423f-83d5-b474ae465154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.808444 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa2235-9eb4-423f-83d5-b474ae465154 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.808723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa2235-9eb4-423f-83d5-b474ae465154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.808998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa2235-9eb4-423f-83d5-b474ae465154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.809764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa2235-9eb4-423f-83d5-b474ae465154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.811980 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa2235-9eb4-423f-83d5-b474ae465154 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.822018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1365/2717] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:26.822842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa2235-9eb4-423f-83d5-b474ae465154 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:26.824468 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ecfa2235-9eb4-423f-83d5-b474ae465154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:26.863066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ab7fe-3671-4614-8993-2ee756a2e4f3 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.867224 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce6462-9b84-4896-a930-71c6bd7900a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.867986 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce6462-9b84-4896-a930-71c6bd7900a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.868400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce6462-9b84-4896-a930-71c6bd7900a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.868784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce6462-9b84-4896-a930-71c6bd7900a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.878902 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce6462-9b84-4896-a930-71c6bd7900a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.880357 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce6462-9b84-4896-a930-71c6bd7900a4 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.884534 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa2235-9eb4-423f-83d5-b474ae465154 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.888206 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d32cb2-4b34-4e20-8b6e-f6fe1fa81348 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.888847 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d32cb2-4b34-4e20-8b6e-f6fe1fa81348 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.889717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d32cb2-4b34-4e20-8b6e-f6fe1fa81348 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.889998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d32cb2-4b34-4e20-8b6e-f6fe1fa81348 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.890665 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1353/2718] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:20:26.891494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce6462-9b84-4896-a930-71c6bd7900a4 service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:26.891862 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c2d32cb2-4b34-4e20-8b6e-f6fe1fa81348 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:26.900642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce6462-9b84-4896-a930-71c6bd7900a4 service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.905513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d7e1f-8996-437f-acec-ac8a0d1d6316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.906200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d7e1f-8996-437f-acec-ac8a0d1d6316 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.906489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d7e1f-8996-437f-acec-ac8a0d1d6316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.906812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d7e1f-8996-437f-acec-ac8a0d1d6316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.907540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d7e1f-8996-437f-acec-ac8a0d1d6316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.909127 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d7e1f-8996-437f-acec-ac8a0d1d6316 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.919443 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1354/2719] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:26.923545 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d7e1f-8996-437f-acec-ac8a0d1d6316 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:26.933365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d7e1f-8996-437f-acec-ac8a0d1d6316 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:26.940035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378d3cf-7c1a-4310-b888-d3eefc2403ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:26.941186 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378d3cf-7c1a-4310-b888-d3eefc2403ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.941312 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378d3cf-7c1a-4310-b888-d3eefc2403ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.941768 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378d3cf-7c1a-4310-b888-d3eefc2403ff admin admin] PATH_INFO: `/v3/projects/89985f3bd3184d10a94b167827b39a3f/users/c6cd8828e7b140569aaea330051b00d6/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:26.955843 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d32cb2-4b34-4e20-8b6e-f6fe1fa81348 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.957444 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d32cb2-4b34-4e20-8b6e-f6fe1fa81348 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.961371 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378d3cf-7c1a-4310-b888-d3eefc2403ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=89985f3bd3184d10a94b167827b39a3f, user_id=c6cd8828e7b140569aaea330051b00d6, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:26.965188 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378d3cf-7c1a-4310-b888-d3eefc2403ff admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:26.967733 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1366/2720] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:26.980816 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1355/2721] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:26 2022] PUT /identity/v3/projects/89985f3bd3184d10a94b167827b39a3f/users/c6cd8828e7b140569aaea330051b00d6/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:20:26.986933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2966e7-2a02-4e3b-ac07-4da31c6128a5 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:26.987299 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2966e7-2a02-4e3b-ac07-4da31c6128a5 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:26.987654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2966e7-2a02-4e3b-ac07-4da31c6128a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:27.001078 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:27.010408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-fa2966e7-2a02-4e3b-ac07-4da31c6128a5 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:27.011768 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-fa2966e7-2a02-4e3b-ac07-4da31c6128a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:27.084264 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1367/2722] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:27.097503 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378d3cf-7c1a-4310-b888-d3eefc2403ff admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:27.168941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378d3cf-7c1a-4310-b888-d3eefc2403ff admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:27.173388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbd682-4c0b-4095-b6c1-4f6d9df9a770 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:27.174535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbd682-4c0b-4095-b6c1-4f6d9df9a770 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:27.174535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbd682-4c0b-4095-b6c1-4f6d9df9a770 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:27.174830 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbd682-4c0b-4095-b6c1-4f6d9df9a770 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:27.185357 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbd682-4c0b-4095-b6c1-4f6d9df9a770 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:27.187470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbd682-4c0b-4095-b6c1-4f6d9df9a770 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:27.197446 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1356/2723] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:27.219840 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2966e7-2a02-4e3b-ac07-4da31c6128a5 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:27.221445 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-fa2966e7-2a02-4e3b-ac07-4da31c6128a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:27.234581 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2966e7-2a02-4e3b-ac07-4da31c6128a5 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:27.239827 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc0935-d880-45a1-b4d6-3428df02ec9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:27.240915 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc0935-d880-45a1-b4d6-3428df02ec9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:27.241224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc0935-d880-45a1-b4d6-3428df02ec9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:27.241527 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc0935-d880-45a1-b4d6-3428df02ec9b admin admin] PATH_INFO: `/v3/projects/561446eabd144133bed76ba2291d392e` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:27.242770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc0935-d880-45a1-b4d6-3428df02ec9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=561446eabd144133bed76ba2291d392e)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:27.244680 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc0935-d880-45a1-b4d6-3428df02ec9b admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:27.314658 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-66bc0935-d880-45a1-b4d6-3428df02ec9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '561446eabd144133bed76ba2291d392e', 'request_id': 'req-66bc0935-d880-45a1-b4d6-3428df02ec9b'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:27.321377 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-66bc0935-d880-45a1-b4d6-3428df02ec9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '561446eabd144133bed76ba2291d392e', 'request_id': 'req-66bc0935-d880-45a1-b4d6-3428df02ec9b'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:27.329180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-66bc0935-d880-45a1-b4d6-3428df02ec9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '561446eabd144133bed76ba2291d392e', 'request_id': 'req-66bc0935-d880-45a1-b4d6-3428df02ec9b'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:27.345872 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-66bc0935-d880-45a1-b4d6-3428df02ec9b admin admin] The token cache is being invalidate because project 561446eabd144133bed76ba2291d392e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:27.346762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-66bc0935-d880-45a1-b4d6-3428df02ec9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:27.347674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-66bc0935-d880-45a1-b4d6-3428df02ec9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:27.349716 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1368/2724] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:27 2022] DELETE /identity/v3/projects/561446eabd144133bed76ba2291d392e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:20:27.353535 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbd682-4c0b-4095-b6c1-4f6d9df9a770 service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:27.365012 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbd682-4c0b-4095-b6c1-4f6d9df9a770 service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:27.407923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:27.435559 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1758d34-ac62-4f7a-b2f8-241710c2393b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:27.437108 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1758d34-ac62-4f7a-b2f8-241710c2393b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:27.437396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1758d34-ac62-4f7a-b2f8-241710c2393b admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:27.437676 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1758d34-ac62-4f7a-b2f8-241710c2393b admin admin] PATH_INFO: `/v3/users/34ea89f50eab41e4a3e11d784407b717` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:27.438948 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1758d34-ac62-4f7a-b2f8-241710c2393b admin admin] RBAC: Authorizing `identity:delete_user(user_id=34ea89f50eab41e4a3e11d784407b717)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:27.440896 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1758d34-ac62-4f7a-b2f8-241710c2393b admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:27.491401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f1758d34-ac62-4f7a-b2f8-241710c2393b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34ea89f50eab41e4a3e11d784407b717'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:27.504680 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f1758d34-ac62-4f7a-b2f8-241710c2393b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34ea89f50eab41e4a3e11d784407b717'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:27.513072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f1758d34-ac62-4f7a-b2f8-241710c2393b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34ea89f50eab41e4a3e11d784407b717'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:27.513827 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f1758d34-ac62-4f7a-b2f8-241710c2393b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34ea89f50eab41e4a3e11d784407b717'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:27.521327 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f1758d34-ac62-4f7a-b2f8-241710c2393b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34ea89f50eab41e4a3e11d784407b717'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:27.527404 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1357/2725] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:20:27 2022] DELETE /identity/v3/users/34ea89f50eab41e4a3e11d784407b717 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:27.946431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc0935-d880-45a1-b4d6-3428df02ec9b admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:27.948068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-66bc0935-d880-45a1-b4d6-3428df02ec9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:27.965632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc0935-d880-45a1-b4d6-3428df02ec9b admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:28.017224 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6326b8-fb6e-4aee-818c-d41b4f6b5fc8 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:28.017756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6326b8-fb6e-4aee-818c-d41b4f6b5fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:28.018714 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6326b8-fb6e-4aee-818c-d41b4f6b5fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:28.040729 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a1810-4d2d-49f8-bd00-0f64726a01d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:28.041596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a1810-4d2d-49f8-bd00-0f64726a01d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:28.041883 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a1810-4d2d-49f8-bd00-0f64726a01d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:28.042248 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a1810-4d2d-49f8-bd00-0f64726a01d5 admin admin] PATH_INFO: `/v3/projects/1262739df7864a03a8a2454c0b0f7285` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:28.043550 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a1810-4d2d-49f8-bd00-0f64726a01d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1262739df7864a03a8a2454c0b0f7285)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:28.045607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a1810-4d2d-49f8-bd00-0f64726a01d5 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:28.047864 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-8c6326b8-fb6e-4aee-818c-d41b4f6b5fc8 None None] MFA Rules not processed for user `c6cd8828e7b140569aaea330051b00d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:28.049950 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8c6326b8-fb6e-4aee-818c-d41b4f6b5fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:28.105533 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:28.120639 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-366a1810-4d2d-49f8-bd00-0f64726a01d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1262739df7864a03a8a2454c0b0f7285', 'request_id': 'req-366a1810-4d2d-49f8-bd00-0f64726a01d5'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:28.129289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-366a1810-4d2d-49f8-bd00-0f64726a01d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1262739df7864a03a8a2454c0b0f7285', 'request_id': 'req-366a1810-4d2d-49f8-bd00-0f64726a01d5'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:28.139624 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-366a1810-4d2d-49f8-bd00-0f64726a01d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1262739df7864a03a8a2454c0b0f7285', 'request_id': 'req-366a1810-4d2d-49f8-bd00-0f64726a01d5'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:28.153517 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1358/2726] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:28.162632 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-366a1810-4d2d-49f8-bd00-0f64726a01d5 admin admin] The token cache is being invalidate because project 1262739df7864a03a8a2454c0b0f7285 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:28.163532 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-366a1810-4d2d-49f8-bd00-0f64726a01d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:28.164745 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-366a1810-4d2d-49f8-bd00-0f64726a01d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:28.167822 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6326b8-fb6e-4aee-818c-d41b4f6b5fc8 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:28.167991 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1369/2727] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:27 2022] DELETE /identity/v3/projects/1262739df7864a03a8a2454c0b0f7285 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:28.169732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8c6326b8-fb6e-4aee-818c-d41b4f6b5fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:28.174300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a8f53-e320-4a8f-a8df-5df0fa4d76f2 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:28.174913 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a8f53-e320-4a8f-a8df-5df0fa4d76f2 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:28.175441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a8f53-e320-4a8f-a8df-5df0fa4d76f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:28.201026 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-713a8f53-e320-4a8f-a8df-5df0fa4d76f2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:28.203871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-713a8f53-e320-4a8f-a8df-5df0fa4d76f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:28.233141 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:28.241739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6326b8-fb6e-4aee-818c-d41b4f6b5fc8 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:28.286039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1370/2728] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:28.294073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d5a91-b85c-411a-b560-69dabfb40df0 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:28.294372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d5a91-b85c-411a-b560-69dabfb40df0 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:28.295079 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d5a91-b85c-411a-b560-69dabfb40df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:28.305700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745e192-d095-4502-93aa-1645f28435ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:28.306356 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745e192-d095-4502-93aa-1645f28435ff service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:28.306638 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745e192-d095-4502-93aa-1645f28435ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:28.306955 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745e192-d095-4502-93aa-1645f28435ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:28.308730 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-3745e192-d095-4502-93aa-1645f28435ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:28.313326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-5c7d5a91-b85c-411a-b560-69dabfb40df0 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:28.314486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5c7d5a91-b85c-411a-b560-69dabfb40df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:28.322241 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1371/2729] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:20:28.327549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d5a91-b85c-411a-b560-69dabfb40df0 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:28.336510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d5a91-b85c-411a-b560-69dabfb40df0 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:28.341047 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ced78b-d7c5-4966-9545-409e074d9428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:28.341689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ced78b-d7c5-4966-9545-409e074d9428 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:28.341970 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ced78b-d7c5-4966-9545-409e074d9428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:28.342408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ced78b-d7c5-4966-9545-409e074d9428 admin admin] PATH_INFO: `/v3/domains` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:28.343376 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ced78b-d7c5-4966-9545-409e074d9428 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80333) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:20:28.343684 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ced78b-d7c5-4966-9545-409e074d9428 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:28.345254 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ced78b-d7c5-4966-9545-409e074d9428 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:28.362375 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1372/2730] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 09:20:28.366349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ced78b-d7c5-4966-9545-409e074d9428 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:28.376845 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745e192-d095-4502-93aa-1645f28435ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:28.378302 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745e192-d095-4502-93aa-1645f28435ff service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:28.384959 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ced78b-d7c5-4966-9545-409e074d9428 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:28.389856 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1359/2731] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:20:28.391238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6a789-e3bb-47ae-bfd1-6317fb97a0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:28.392020 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6a789-e3bb-47ae-bfd1-6317fb97a0a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:28.392297 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6a789-e3bb-47ae-bfd1-6317fb97a0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:28.392584 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6a789-e3bb-47ae-bfd1-6317fb97a0a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:28.393336 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6a789-e3bb-47ae-bfd1-6317fb97a0a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:28.395285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6a789-e3bb-47ae-bfd1-6317fb97a0a5 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:28.427333 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1373/2732] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:20:28 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:20:28.431323 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745e192-d095-4502-93aa-1645f28435ff service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:28.443253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745e192-d095-4502-93aa-1645f28435ff service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:28.521548 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c95e9-7843-4407-9c10-d8e21667312e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:28.522610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c95e9-7843-4407-9c10-d8e21667312e admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:28.522947 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c95e9-7843-4407-9c10-d8e21667312e admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:28.523362 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c95e9-7843-4407-9c10-d8e21667312e admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:28.524077 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6c95e9-7843-4407-9c10-d8e21667312e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:28.530185 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6c95e9-7843-4407-9c10-d8e21667312e admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:28.556239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1360/2733] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:20:28 2022] POST /identity/v3/users => generated 283 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:28.558023 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6a789-e3bb-47ae-bfd1-6317fb97a0a5 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:28.572712 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6a789-e3bb-47ae-bfd1-6317fb97a0a5 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:28.579048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178b270-f621-4a4f-80a4-2d7336c6a7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:28.579800 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178b270-f621-4a4f-80a4-2d7336c6a7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:28.580095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178b270-f621-4a4f-80a4-2d7336c6a7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:28.580440 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178b270-f621-4a4f-80a4-2d7336c6a7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:28.581189 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178b270-f621-4a4f-80a4-2d7336c6a7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:28.582796 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178b270-f621-4a4f-80a4-2d7336c6a7b6 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:28.593858 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1374/2734] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:28.598026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c95e9-7843-4407-9c10-d8e21667312e admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:28.609962 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c95e9-7843-4407-9c10-d8e21667312e admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:28.614668 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128b5e1-8ee1-4333-add5-56d7867d7f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:28.615361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128b5e1-8ee1-4333-add5-56d7867d7f77 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:28.615647 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128b5e1-8ee1-4333-add5-56d7867d7f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:28.615929 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128b5e1-8ee1-4333-add5-56d7867d7f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:28.616660 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128b5e1-8ee1-4333-add5-56d7867d7f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:28.618244 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128b5e1-8ee1-4333-add5-56d7867d7f77 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:28.631027 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1361/2735] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:28.635595 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178b270-f621-4a4f-80a4-2d7336c6a7b6 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:28.646640 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178b270-f621-4a4f-80a4-2d7336c6a7b6 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:28.651807 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82ce9a-d298-4ab5-b489-0fca058db5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:28.652722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82ce9a-d298-4ab5-b489-0fca058db5ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:28.653002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82ce9a-d298-4ab5-b489-0fca058db5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:28.653280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82ce9a-d298-4ab5-b489-0fca058db5ab admin admin] PATH_INFO: `/v3/projects/96d9b70aa0b841d98292310b6fb98a43/users/4f0c8d10014a4c30a388144cf5860b5e/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:28.669410 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82ce9a-d298-4ab5-b489-0fca058db5ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=96d9b70aa0b841d98292310b6fb98a43, user_id=4f0c8d10014a4c30a388144cf5860b5e, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:28.672789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af82ce9a-d298-4ab5-b489-0fca058db5ab admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:28.688773 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1375/2736] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:28 2022] PUT /identity/v3/projects/96d9b70aa0b841d98292310b6fb98a43/users/4f0c8d10014a4c30a388144cf5860b5e/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:28.700357 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8820ac-ef5b-436d-8cef-2f6d4c4efa64 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:28.700662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8820ac-ef5b-436d-8cef-2f6d4c4efa64 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:28.700946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8820ac-ef5b-436d-8cef-2f6d4c4efa64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:28.729211 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-4c8820ac-ef5b-436d-8cef-2f6d4c4efa64 None None] MFA Rules not processed for user `4f0c8d10014a4c30a388144cf5860b5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:28.730564 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4c8820ac-ef5b-436d-8cef-2f6d4c4efa64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:28.756821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:28.802396 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1362/2737] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:28.806782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82ce9a-d298-4ab5-b489-0fca058db5ab admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:28.852792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:28.870405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af82ce9a-d298-4ab5-b489-0fca058db5ab admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:28.875263 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade710e8-22d2-4c94-92c2-24af7085cfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:28.875928 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade710e8-22d2-4c94-92c2-24af7085cfd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:28.876214 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade710e8-22d2-4c94-92c2-24af7085cfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:28.876494 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade710e8-22d2-4c94-92c2-24af7085cfd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:28.877199 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade710e8-22d2-4c94-92c2-24af7085cfd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:28.879114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade710e8-22d2-4c94-92c2-24af7085cfd0 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:28.903776 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1376/2738] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:20:28 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:28.908468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8820ac-ef5b-436d-8cef-2f6d4c4efa64 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:28.919073 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8820ac-ef5b-436d-8cef-2f6d4c4efa64 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:28.975985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12916ee4-72c1-4608-9ff7-ec13a4a11c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:28.976836 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12916ee4-72c1-4608-9ff7-ec13a4a11c78 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:28.976998 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12916ee4-72c1-4608-9ff7-ec13a4a11c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:28.977280 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12916ee4-72c1-4608-9ff7-ec13a4a11c78 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:28.978065 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12916ee4-72c1-4608-9ff7-ec13a4a11c78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:28.980044 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12916ee4-72c1-4608-9ff7-ec13a4a11c78 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:28.999952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1363/2739] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:20:28 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:29.003877 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade710e8-22d2-4c94-92c2-24af7085cfd0 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:29.014600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade710e8-22d2-4c94-92c2-24af7085cfd0 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:29.019279 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2032f5-e867-4a57-940d-7b1b46111280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:29.019931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2032f5-e867-4a57-940d-7b1b46111280 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:29.020215 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2032f5-e867-4a57-940d-7b1b46111280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:29.020490 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2032f5-e867-4a57-940d-7b1b46111280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:29.021202 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2032f5-e867-4a57-940d-7b1b46111280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:29.022862 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2032f5-e867-4a57-940d-7b1b46111280 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:29.033216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1377/2740] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:29.037237 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12916ee4-72c1-4608-9ff7-ec13a4a11c78 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:29.047502 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12916ee4-72c1-4608-9ff7-ec13a4a11c78 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:29.052319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756586d-0649-426e-8d05-e3ac7b7b34de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:29.053194 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756586d-0649-426e-8d05-e3ac7b7b34de admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:29.053476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756586d-0649-426e-8d05-e3ac7b7b34de admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:29.053753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756586d-0649-426e-8d05-e3ac7b7b34de admin admin] PATH_INFO: `/v3/projects/99ca62efc529479f8701384dd6f6f14c/users/e2d6984f6f8e470b876ad5930fa15a59/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:29.069685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756586d-0649-426e-8d05-e3ac7b7b34de admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ca62efc529479f8701384dd6f6f14c, user_id=e2d6984f6f8e470b876ad5930fa15a59, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:29.073185 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756586d-0649-426e-8d05-e3ac7b7b34de admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:29.086880 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1364/2741] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:29 2022] PUT /identity/v3/projects/99ca62efc529479f8701384dd6f6f14c/users/e2d6984f6f8e470b876ad5930fa15a59/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:29.090871 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2032f5-e867-4a57-940d-7b1b46111280 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:29.100884 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2032f5-e867-4a57-940d-7b1b46111280 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:29.156554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82eaf4-6f5f-4639-89d3-4847e2fe5f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:29.157158 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82eaf4-6f5f-4639-89d3-4847e2fe5f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:29.157422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82eaf4-6f5f-4639-89d3-4847e2fe5f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:29.157703 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82eaf4-6f5f-4639-89d3-4847e2fe5f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:29.158432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82eaf4-6f5f-4639-89d3-4847e2fe5f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:29.160051 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82eaf4-6f5f-4639-89d3-4847e2fe5f73 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:29.169879 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1378/2742] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:29.174550 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756586d-0649-426e-8d05-e3ac7b7b34de admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:29.184736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756586d-0649-426e-8d05-e3ac7b7b34de admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:29.189911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca00cfa-6a63-4620-9966-f1439f035042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:29.190827 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca00cfa-6a63-4620-9966-f1439f035042 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:29.191111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca00cfa-6a63-4620-9966-f1439f035042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:29.191388 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca00cfa-6a63-4620-9966-f1439f035042 admin admin] PATH_INFO: `/v3/projects/99ca62efc529479f8701384dd6f6f14c/users/e2d6984f6f8e470b876ad5930fa15a59/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:29.193365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca00cfa-6a63-4620-9966-f1439f035042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ca62efc529479f8701384dd6f6f14c, user_id=e2d6984f6f8e470b876ad5930fa15a59, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:29.196023 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca00cfa-6a63-4620-9966-f1439f035042 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:29.207289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1365/2743] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:29 2022] PUT /identity/v3/projects/99ca62efc529479f8701384dd6f6f14c/users/e2d6984f6f8e470b876ad5930fa15a59/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:29.217386 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d78bd-a320-4fb3-94e5-8291b2094a9e None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:29.217679 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d78bd-a320-4fb3-94e5-8291b2094a9e None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:29.217961 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d78bd-a320-4fb3-94e5-8291b2094a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:29.243812 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-3c8d78bd-a320-4fb3-94e5-8291b2094a9e None None] MFA Rules not processed for user `e2d6984f6f8e470b876ad5930fa15a59`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:29.245030 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-3c8d78bd-a320-4fb3-94e5-8291b2094a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:29.315311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1379/2744] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:29 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:29.328489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca00cfa-6a63-4620-9966-f1439f035042 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:29.330188 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-2ca00cfa-6a63-4620-9966-f1439f035042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:29.387569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca00cfa-6a63-4620-9966-f1439f035042 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:29.391224 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2c22b-3fd2-48f0-badb-52b6fc1d2507 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:29.391861 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2c22b-3fd2-48f0-badb-52b6fc1d2507 service cinder] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:29.392142 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2c22b-3fd2-48f0-badb-52b6fc1d2507 service cinder] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:29.392417 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2c22b-3fd2-48f0-badb-52b6fc1d2507 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:29.401314 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2c22b-3fd2-48f0-badb-52b6fc1d2507 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:29.402762 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2c22b-3fd2-48f0-badb-52b6fc1d2507 service cinder] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:29.412768 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1366/2745] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:29 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:20:29.455488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d78bd-a320-4fb3-94e5-8291b2094a9e None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:29.456883 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-3c8d78bd-a320-4fb3-94e5-8291b2094a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:29.469042 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d78bd-a320-4fb3-94e5-8291b2094a9e None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:29.472553 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df948c7-0e26-4cdd-9995-1d9899a78c1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:29.473190 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df948c7-0e26-4cdd-9995-1d9899a78c1b service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:29.473471 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df948c7-0e26-4cdd-9995-1d9899a78c1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:29.473746 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df948c7-0e26-4cdd-9995-1d9899a78c1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:29.512412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:29.531429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df948c7-0e26-4cdd-9995-1d9899a78c1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:29.532851 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df948c7-0e26-4cdd-9995-1d9899a78c1b service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:29.541661 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1380/2746] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:29.613268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2c22b-3fd2-48f0-badb-52b6fc1d2507 service cinder] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:29.614724 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-72c2c22b-3fd2-48f0-badb-52b6fc1d2507 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:29.637593 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:29.685720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2c22b-3fd2-48f0-badb-52b6fc1d2507 service cinder] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:29.691837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1c7b3-54af-46de-87a4-977e944349ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:29.692826 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1c7b3-54af-46de-87a4-977e944349ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:29.693257 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1c7b3-54af-46de-87a4-977e944349ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:29.693525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1c7b3-54af-46de-87a4-977e944349ee admin admin] PATH_INFO: `/v3/users/3733fa2e042a4596aef68a01de090293` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:29.695168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1c7b3-54af-46de-87a4-977e944349ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=3733fa2e042a4596aef68a01de090293)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:29.697538 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1c7b3-54af-46de-87a4-977e944349ee admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:29.749983 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-66f1c7b3-54af-46de-87a4-977e944349ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3733fa2e042a4596aef68a01de090293'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:29.763330 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-66f1c7b3-54af-46de-87a4-977e944349ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3733fa2e042a4596aef68a01de090293'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:29.783379 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-66f1c7b3-54af-46de-87a4-977e944349ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3733fa2e042a4596aef68a01de090293'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:29.784390 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-66f1c7b3-54af-46de-87a4-977e944349ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3733fa2e042a4596aef68a01de090293'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:29.793618 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-66f1c7b3-54af-46de-87a4-977e944349ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3733fa2e042a4596aef68a01de090293'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:29.799716 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cdf67-a26d-4afb-807a-88d365c47659 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:29.800277 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cdf67-a26d-4afb-807a-88d365c47659 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:29.800623 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cdf67-a26d-4afb-807a-88d365c47659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:29.817615 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1367/2747] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:20:29 2022] DELETE /identity/v3/users/3733fa2e042a4596aef68a01de090293 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:29.819338 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c4498-fdf0-45bb-8de4-84a182eaff8a None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:29.819631 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c4498-fdf0-45bb-8de4-84a182eaff8a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:29.819925 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c4498-fdf0-45bb-8de4-84a182eaff8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:29.825482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-357cdf67-a26d-4afb-807a-88d365c47659 None None] MFA Rules not processed for user `c6cd8828e7b140569aaea330051b00d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:29.827228 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-357cdf67-a26d-4afb-807a-88d365c47659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:29.843455 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-e73c4498-fdf0-45bb-8de4-84a182eaff8a None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:29.844789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e73c4498-fdf0-45bb-8de4-84a182eaff8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:29.907192 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1381/2748] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:20:29.912365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1368/2749] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:29.922078 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357cdf67-a26d-4afb-807a-88d365c47659 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:29.923502 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-357cdf67-a26d-4afb-807a-88d365c47659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:29.985347 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357cdf67-a26d-4afb-807a-88d365c47659 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:29.989059 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802ab41-85b8-4984-ad66-6d62c9ecdffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:29.989822 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802ab41-85b8-4984-ad66-6d62c9ecdffc service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:29.990203 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802ab41-85b8-4984-ad66-6d62c9ecdffc service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:29.990578 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802ab41-85b8-4984-ad66-6d62c9ecdffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:30.000560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802ab41-85b8-4984-ad66-6d62c9ecdffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:30.002019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802ab41-85b8-4984-ad66-6d62c9ecdffc service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:30.011787 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1382/2750] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:30.189268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c4498-fdf0-45bb-8de4-84a182eaff8a None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:30.198788 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c4498-fdf0-45bb-8de4-84a182eaff8a None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:30.203000 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335dc037-53fa-4f2b-b281-87006db9fb1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:30.203753 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335dc037-53fa-4f2b-b281-87006db9fb1d service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:30.203957 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335dc037-53fa-4f2b-b281-87006db9fb1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:30.204249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335dc037-53fa-4f2b-b281-87006db9fb1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:30.206238 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-335dc037-53fa-4f2b-b281-87006db9fb1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:30.216423 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335dc037-53fa-4f2b-b281-87006db9fb1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:30.218574 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335dc037-53fa-4f2b-b281-87006db9fb1d service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:30.228253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1369/2751] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:30.670111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802ab41-85b8-4984-ad66-6d62c9ecdffc service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:30.672704 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-c802ab41-85b8-4984-ad66-6d62c9ecdffc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:30.687205 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802ab41-85b8-4984-ad66-6d62c9ecdffc service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:30.691906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:30.693532 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:30.693532 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:30.693532 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca admin admin] PATH_INFO: `/v3/projects/6af8a41843f0451e8fe98bb4fa37218a` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:30.695285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=6af8a41843f0451e8fe98bb4fa37218a)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:30.697419 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:30.748607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6af8a41843f0451e8fe98bb4fa37218a', 'request_id': 'req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:30.754479 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6af8a41843f0451e8fe98bb4fa37218a', 'request_id': 'req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:30.761224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6af8a41843f0451e8fe98bb4fa37218a', 'request_id': 'req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:30.775587 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca admin admin] The token cache is being invalidate because project 6af8a41843f0451e8fe98bb4fa37218a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:30.775957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:30.776716 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:30.778410 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1383/2752] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:30 2022] DELETE /identity/v3/projects/6af8a41843f0451e8fe98bb4fa37218a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:30.783492 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335dc037-53fa-4f2b-b281-87006db9fb1d service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:30.802630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335dc037-53fa-4f2b-b281-87006db9fb1d service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:30.829498 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:30.872026 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9fabc-465d-4eac-8bfe-ca85d241d1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:30.872916 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9fabc-465d-4eac-8bfe-ca85d241d1f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:30.873242 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9fabc-465d-4eac-8bfe-ca85d241d1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:30.873659 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9fabc-465d-4eac-8bfe-ca85d241d1f5 admin admin] PATH_INFO: `/v3/users/1a0fe805451844ab8e55bb772e1f5f2d` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:30.874952 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9fabc-465d-4eac-8bfe-ca85d241d1f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a0fe805451844ab8e55bb772e1f5f2d)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:30.876908 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9fabc-465d-4eac-8bfe-ca85d241d1f5 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:30.938321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-12b9fabc-465d-4eac-8bfe-ca85d241d1f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a0fe805451844ab8e55bb772e1f5f2d'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:30.953601 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-12b9fabc-465d-4eac-8bfe-ca85d241d1f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a0fe805451844ab8e55bb772e1f5f2d'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:30.965862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-12b9fabc-465d-4eac-8bfe-ca85d241d1f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a0fe805451844ab8e55bb772e1f5f2d'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:30.966646 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-12b9fabc-465d-4eac-8bfe-ca85d241d1f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a0fe805451844ab8e55bb772e1f5f2d'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:30.976834 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-12b9fabc-465d-4eac-8bfe-ca85d241d1f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a0fe805451844ab8e55bb772e1f5f2d'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:30.984426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1370/2753] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:20:30 2022] DELETE /identity/v3/users/1a0fe805451844ab8e55bb772e1f5f2d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:31.287701 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:31.301353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:31.324548 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2e402-ad87-4fdd-a510-ed28aa1a60ca admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:31.330739 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:31.391265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107ec17-9538-4b5d-8785-9f176844601f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:31.392307 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107ec17-9538-4b5d-8785-9f176844601f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:31.392604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107ec17-9538-4b5d-8785-9f176844601f admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:31.392878 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107ec17-9538-4b5d-8785-9f176844601f admin admin] PATH_INFO: `/v3/projects/ab02a6020b504783a88d734c329f9330` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:31.394111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107ec17-9538-4b5d-8785-9f176844601f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab02a6020b504783a88d734c329f9330)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:31.396150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107ec17-9538-4b5d-8785-9f176844601f admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:31.445040 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6107ec17-9538-4b5d-8785-9f176844601f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab02a6020b504783a88d734c329f9330', 'request_id': 'req-6107ec17-9538-4b5d-8785-9f176844601f'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:31.450849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6107ec17-9538-4b5d-8785-9f176844601f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab02a6020b504783a88d734c329f9330', 'request_id': 'req-6107ec17-9538-4b5d-8785-9f176844601f'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:31.457505 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6107ec17-9538-4b5d-8785-9f176844601f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab02a6020b504783a88d734c329f9330', 'request_id': 'req-6107ec17-9538-4b5d-8785-9f176844601f'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:31.470048 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6107ec17-9538-4b5d-8785-9f176844601f admin admin] The token cache is being invalidate because project ab02a6020b504783a88d734c329f9330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:31.470362 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6107ec17-9538-4b5d-8785-9f176844601f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:31.471013 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-6107ec17-9538-4b5d-8785-9f176844601f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:31.472783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1384/2754] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:31 2022] DELETE /identity/v3/projects/ab02a6020b504783a88d734c329f9330 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:31.479061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76761414-d115-473f-a0f0-7265e7101f6b None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:31.479297 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76761414-d115-473f-a0f0-7265e7101f6b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:31.479581 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76761414-d115-473f-a0f0-7265e7101f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:31.497826 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-76761414-d115-473f-a0f0-7265e7101f6b None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:31.499588 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-76761414-d115-473f-a0f0-7265e7101f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:31.566479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1371/2755] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:31.574405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f496312-ddc1-42cf-a97e-467a7564f37f None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:31.574733 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f496312-ddc1-42cf-a97e-467a7564f37f None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:31.575282 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f496312-ddc1-42cf-a97e-467a7564f37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:31.596197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-8f496312-ddc1-42cf-a97e-467a7564f37f None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:31.597719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8f496312-ddc1-42cf-a97e-467a7564f37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:31.608604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1385/2756] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:31.612951 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76761414-d115-473f-a0f0-7265e7101f6b None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:31.623093 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76761414-d115-473f-a0f0-7265e7101f6b None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:31.629098 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b380f7-da79-4535-b141-88ef86b94fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:31.629781 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b380f7-da79-4535-b141-88ef86b94fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:31.630057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b380f7-da79-4535-b141-88ef86b94fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:31.630336 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b380f7-da79-4535-b141-88ef86b94fd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:31.631133 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b380f7-da79-4535-b141-88ef86b94fd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:20:31.631421 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b380f7-da79-4535-b141-88ef86b94fd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:31.633750 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b380f7-da79-4535-b141-88ef86b94fd0 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:31.646211 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1372/2757] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:20:31.650544 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f496312-ddc1-42cf-a97e-467a7564f37f None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:31.660748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f496312-ddc1-42cf-a97e-467a7564f37f None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:31.666366 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b0d6a-7e8c-4296-8fba-3cb33ec9b738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:31.667216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b0d6a-7e8c-4296-8fba-3cb33ec9b738 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:31.667509 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b0d6a-7e8c-4296-8fba-3cb33ec9b738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:31.667791 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b0d6a-7e8c-4296-8fba-3cb33ec9b738 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:31.668505 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b0d6a-7e8c-4296-8fba-3cb33ec9b738 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:31.671334 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b0d6a-7e8c-4296-8fba-3cb33ec9b738 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:31.706863 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1386/2758] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:20:31 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:20:31.708353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b380f7-da79-4535-b141-88ef86b94fd0 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:31.721321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b380f7-da79-4535-b141-88ef86b94fd0 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:31.760547 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:31.799654 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49e1d2-af54-4a54-b379-ee0c67a9600c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:31.800365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49e1d2-af54-4a54-b379-ee0c67a9600c admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:31.800640 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49e1d2-af54-4a54-b379-ee0c67a9600c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:31.800895 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49e1d2-af54-4a54-b379-ee0c67a9600c admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:31.801591 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49e1d2-af54-4a54-b379-ee0c67a9600c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:31.803696 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49e1d2-af54-4a54-b379-ee0c67a9600c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:31.827832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1373/2759] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:20:31 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:20:31.831895 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b0d6a-7e8c-4296-8fba-3cb33ec9b738 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:31.842869 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b0d6a-7e8c-4296-8fba-3cb33ec9b738 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:31.847533 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811469c-8b5b-44c2-9acd-ae983417e86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:31.848208 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811469c-8b5b-44c2-9acd-ae983417e86f admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:31.848492 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811469c-8b5b-44c2-9acd-ae983417e86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:31.848769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811469c-8b5b-44c2-9acd-ae983417e86f admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:31.849514 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811469c-8b5b-44c2-9acd-ae983417e86f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:31.851143 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811469c-8b5b-44c2-9acd-ae983417e86f admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:31.862122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1387/2760] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:31.866187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49e1d2-af54-4a54-b379-ee0c67a9600c admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:31.877057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49e1d2-af54-4a54-b379-ee0c67a9600c admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:31.882344 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe8ccb-131b-4cae-afdd-a816cd67388a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:31.883035 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe8ccb-131b-4cae-afdd-a816cd67388a admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:31.884121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe8ccb-131b-4cae-afdd-a816cd67388a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:31.884121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe8ccb-131b-4cae-afdd-a816cd67388a admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:31.884346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe8ccb-131b-4cae-afdd-a816cd67388a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:31.885890 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe8ccb-131b-4cae-afdd-a816cd67388a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:31.897081 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1374/2761] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:31.901187 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811469c-8b5b-44c2-9acd-ae983417e86f admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:31.912836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811469c-8b5b-44c2-9acd-ae983417e86f admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:31.917797 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf800e-0057-42d4-bcf8-e7c27e9217f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:31.918697 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf800e-0057-42d4-bcf8-e7c27e9217f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:31.919008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf800e-0057-42d4-bcf8-e7c27e9217f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:31.919290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf800e-0057-42d4-bcf8-e7c27e9217f7 admin admin] PATH_INFO: `/v3/projects/5d4d6988adeb4b32af53259fdd5dbf94/users/bbdcb7eea95b48a8a00327763653a21f/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:31.935986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf800e-0057-42d4-bcf8-e7c27e9217f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d4d6988adeb4b32af53259fdd5dbf94, user_id=bbdcb7eea95b48a8a00327763653a21f, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:31.938933 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf800e-0057-42d4-bcf8-e7c27e9217f7 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:31.953441 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1388/2762] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:31 2022] PUT /identity/v3/projects/5d4d6988adeb4b32af53259fdd5dbf94/users/bbdcb7eea95b48a8a00327763653a21f/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:20:31.964053 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd45bf9-a460-474d-a63b-dd23807290d6 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:31.964349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd45bf9-a460-474d-a63b-dd23807290d6 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:31.964630 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd45bf9-a460-474d-a63b-dd23807290d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:31.992914 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-afd45bf9-a460-474d-a63b-dd23807290d6 None None] MFA Rules not processed for user `bbdcb7eea95b48a8a00327763653a21f`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:31.994058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-afd45bf9-a460-474d-a63b-dd23807290d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:32.062528 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1375/2763] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:32.075269 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf800e-0057-42d4-bcf8-e7c27e9217f7 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:32.077881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e5bf800e-0057-42d4-bcf8-e7c27e9217f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:32.119022 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:32.141480 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf800e-0057-42d4-bcf8-e7c27e9217f7 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:32.145163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588fb10-ac83-4024-a9f2-81c84a637f22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:32.145817 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588fb10-ac83-4024-a9f2-81c84a637f22 service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:32.146119 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588fb10-ac83-4024-a9f2-81c84a637f22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:32.146384 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588fb10-ac83-4024-a9f2-81c84a637f22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:32.155675 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588fb10-ac83-4024-a9f2-81c84a637f22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:32.157129 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588fb10-ac83-4024-a9f2-81c84a637f22 service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:32.166604 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1389/2764] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:33.017454 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd45bf9-a460-474d-a63b-dd23807290d6 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:33.019637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-afd45bf9-a460-474d-a63b-dd23807290d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:33.090739 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd45bf9-a460-474d-a63b-dd23807290d6 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:33.096515 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9751c-9e8c-4b89-9b8e-bb6fe1390bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:33.097393 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9751c-9e8c-4b89-9b8e-bb6fe1390bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:33.097679 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9751c-9e8c-4b89-9b8e-bb6fe1390bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:33.097958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9751c-9e8c-4b89-9b8e-bb6fe1390bcd admin admin] PATH_INFO: `/v3/users/4f0c8d10014a4c30a388144cf5860b5e` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:33.099184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9751c-9e8c-4b89-9b8e-bb6fe1390bcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f0c8d10014a4c30a388144cf5860b5e)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:33.101106 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9751c-9e8c-4b89-9b8e-bb6fe1390bcd admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:33.153047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:33.156342 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bca9751c-9e8c-4b89-9b8e-bb6fe1390bcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f0c8d10014a4c30a388144cf5860b5e'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:33.170182 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bca9751c-9e8c-4b89-9b8e-bb6fe1390bcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f0c8d10014a4c30a388144cf5860b5e'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:33.177567 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bca9751c-9e8c-4b89-9b8e-bb6fe1390bcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f0c8d10014a4c30a388144cf5860b5e'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:33.178595 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bca9751c-9e8c-4b89-9b8e-bb6fe1390bcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f0c8d10014a4c30a388144cf5860b5e'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:33.186144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bca9751c-9e8c-4b89-9b8e-bb6fe1390bcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f0c8d10014a4c30a388144cf5860b5e'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:33.192184 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1376/2765] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:20:33 2022] DELETE /identity/v3/users/4f0c8d10014a4c30a388144cf5860b5e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:33.196802 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4dbb1-7902-4a7e-8317-d0ec2fd96166 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:33.197287 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4dbb1-7902-4a7e-8317-d0ec2fd96166 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:33.197397 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4dbb1-7902-4a7e-8317-d0ec2fd96166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:33.215506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-b8a4dbb1-7902-4a7e-8317-d0ec2fd96166 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:33.216715 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b8a4dbb1-7902-4a7e-8317-d0ec2fd96166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:33.278719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1390/2766] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:33.290637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9751c-9e8c-4b89-9b8e-bb6fe1390bcd admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:33.292103 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-bca9751c-9e8c-4b89-9b8e-bb6fe1390bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:33.349650 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9751c-9e8c-4b89-9b8e-bb6fe1390bcd admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:33.353270 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07198187-fe60-4d5c-b74f-67d8a9c19d0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:33.353926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07198187-fe60-4d5c-b74f-67d8a9c19d0c service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:33.354211 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07198187-fe60-4d5c-b74f-67d8a9c19d0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:33.354489 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07198187-fe60-4d5c-b74f-67d8a9c19d0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:33.364094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07198187-fe60-4d5c-b74f-67d8a9c19d0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:33.365811 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07198187-fe60-4d5c-b74f-67d8a9c19d0c service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:33.375235 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1377/2767] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:33.656285 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4dbb1-7902-4a7e-8317-d0ec2fd96166 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:33.658034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-b8a4dbb1-7902-4a7e-8317-d0ec2fd96166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:33.670375 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4dbb1-7902-4a7e-8317-d0ec2fd96166 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:33.676218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167be05-5557-4e49-937e-d421bbdc7e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:33.677260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167be05-5557-4e49-937e-d421bbdc7e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:33.677530 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167be05-5557-4e49-937e-d421bbdc7e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:33.677816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167be05-5557-4e49-937e-d421bbdc7e0d admin admin] PATH_INFO: `/v3/projects/96d9b70aa0b841d98292310b6fb98a43` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:33.679150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167be05-5557-4e49-937e-d421bbdc7e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=96d9b70aa0b841d98292310b6fb98a43)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:33.681058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167be05-5557-4e49-937e-d421bbdc7e0d admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:33.738753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5167be05-5557-4e49-937e-d421bbdc7e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96d9b70aa0b841d98292310b6fb98a43', 'request_id': 'req-5167be05-5557-4e49-937e-d421bbdc7e0d'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:33.744298 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5167be05-5557-4e49-937e-d421bbdc7e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96d9b70aa0b841d98292310b6fb98a43', 'request_id': 'req-5167be05-5557-4e49-937e-d421bbdc7e0d'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:33.751316 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5167be05-5557-4e49-937e-d421bbdc7e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96d9b70aa0b841d98292310b6fb98a43', 'request_id': 'req-5167be05-5557-4e49-937e-d421bbdc7e0d'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:33.763551 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5167be05-5557-4e49-937e-d421bbdc7e0d admin admin] The token cache is being invalidate because project 96d9b70aa0b841d98292310b6fb98a43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:33.764500 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5167be05-5557-4e49-937e-d421bbdc7e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:33.765218 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-5167be05-5557-4e49-937e-d421bbdc7e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:33.766801 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1391/2768] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:33 2022] DELETE /identity/v3/projects/96d9b70aa0b841d98292310b6fb98a43 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:20:33.772698 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07198187-fe60-4d5c-b74f-67d8a9c19d0c service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:33.783028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07198187-fe60-4d5c-b74f-67d8a9c19d0c service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:33.841794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd6661-1b52-494f-aa0c-359e6f6cc0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:33.842671 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd6661-1b52-494f-aa0c-359e6f6cc0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:33.842983 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd6661-1b52-494f-aa0c-359e6f6cc0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:33.843280 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd6661-1b52-494f-aa0c-359e6f6cc0a2 admin admin] PATH_INFO: `/v3/users/e2d6984f6f8e470b876ad5930fa15a59` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:33.844446 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd6661-1b52-494f-aa0c-359e6f6cc0a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2d6984f6f8e470b876ad5930fa15a59)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:33.846424 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd6661-1b52-494f-aa0c-359e6f6cc0a2 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:33.895672 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8ddd6661-1b52-494f-aa0c-359e6f6cc0a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2d6984f6f8e470b876ad5930fa15a59'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:33.907829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8ddd6661-1b52-494f-aa0c-359e6f6cc0a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2d6984f6f8e470b876ad5930fa15a59'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:33.909049 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:33.916981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8ddd6661-1b52-494f-aa0c-359e6f6cc0a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2d6984f6f8e470b876ad5930fa15a59'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:33.917755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8ddd6661-1b52-494f-aa0c-359e6f6cc0a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2d6984f6f8e470b876ad5930fa15a59'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:33.925504 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8ddd6661-1b52-494f-aa0c-359e6f6cc0a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2d6984f6f8e470b876ad5930fa15a59'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:33.931382 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1378/2769] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:20:33 2022] DELETE /identity/v3/users/e2d6984f6f8e470b876ad5930fa15a59 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:34.365017 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167be05-5557-4e49-937e-d421bbdc7e0d admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:34.366506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-5167be05-5557-4e49-937e-d421bbdc7e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:34.380822 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167be05-5557-4e49-937e-d421bbdc7e0d admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:34.449068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e24fb-bfac-491a-88ef-e13434070928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:34.450405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e24fb-bfac-491a-88ef-e13434070928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:34.450707 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e24fb-bfac-491a-88ef-e13434070928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:34.450991 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e24fb-bfac-491a-88ef-e13434070928 admin admin] PATH_INFO: `/v3/projects/99ca62efc529479f8701384dd6f6f14c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:34.452783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e24fb-bfac-491a-88ef-e13434070928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ca62efc529479f8701384dd6f6f14c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:34.454669 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e24fb-bfac-491a-88ef-e13434070928 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:34.457676 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:34.518731 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-fc0e24fb-bfac-491a-88ef-e13434070928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ca62efc529479f8701384dd6f6f14c', 'request_id': 'req-fc0e24fb-bfac-491a-88ef-e13434070928'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:34.524353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-fc0e24fb-bfac-491a-88ef-e13434070928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ca62efc529479f8701384dd6f6f14c', 'request_id': 'req-fc0e24fb-bfac-491a-88ef-e13434070928'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:34.530551 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-fc0e24fb-bfac-491a-88ef-e13434070928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ca62efc529479f8701384dd6f6f14c', 'request_id': 'req-fc0e24fb-bfac-491a-88ef-e13434070928'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:34.551342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-fc0e24fb-bfac-491a-88ef-e13434070928 admin admin] The token cache is being invalidate because project 99ca62efc529479f8701384dd6f6f14c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:34.551782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-fc0e24fb-bfac-491a-88ef-e13434070928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:34.552665 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-fc0e24fb-bfac-491a-88ef-e13434070928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:34.554637 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1392/2770] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:34 2022] DELETE /identity/v3/projects/99ca62efc529479f8701384dd6f6f14c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:34.559294 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd774a-5559-494d-86a6-aa23f69a6f9a None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:34.559818 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd774a-5559-494d-86a6-aa23f69a6f9a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:34.559924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd774a-5559-494d-86a6-aa23f69a6f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:34.580321 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-e9dd774a-5559-494d-86a6-aa23f69a6f9a None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:34.581547 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e9dd774a-5559-494d-86a6-aa23f69a6f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:34.644104 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1379/2771] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:34.651995 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df8ffd-5418-41a7-9088-591de720c856 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:34.652813 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df8ffd-5418-41a7-9088-591de720c856 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:34.653108 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df8ffd-5418-41a7-9088-591de720c856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:34.672846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-92df8ffd-5418-41a7-9088-591de720c856 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:34.674058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-92df8ffd-5418-41a7-9088-591de720c856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:34.683112 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1393/2772] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:34.688021 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd774a-5559-494d-86a6-aa23f69a6f9a None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:34.697893 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd774a-5559-494d-86a6-aa23f69a6f9a None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:34.702850 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e0afa-cd48-40b6-a0d7-1b8c8bfbca37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:34.703644 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e0afa-cd48-40b6-a0d7-1b8c8bfbca37 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:34.703910 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e0afa-cd48-40b6-a0d7-1b8c8bfbca37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:34.704192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e0afa-cd48-40b6-a0d7-1b8c8bfbca37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:34.704922 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e0afa-cd48-40b6-a0d7-1b8c8bfbca37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:20:34.705233 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e0afa-cd48-40b6-a0d7-1b8c8bfbca37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:34.706855 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e0afa-cd48-40b6-a0d7-1b8c8bfbca37 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:34.718401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1380/2773] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:20:34.722576 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df8ffd-5418-41a7-9088-591de720c856 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:34.731702 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df8ffd-5418-41a7-9088-591de720c856 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:34.737292 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d8233-9d20-4083-94b8-9bf3c2e2b956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:34.737984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d8233-9d20-4083-94b8-9bf3c2e2b956 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:34.738276 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d8233-9d20-4083-94b8-9bf3c2e2b956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:34.738556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d8233-9d20-4083-94b8-9bf3c2e2b956 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:34.739289 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d8233-9d20-4083-94b8-9bf3c2e2b956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:34.741180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d8233-9d20-4083-94b8-9bf3c2e2b956 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:34.761807 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e0afa-cd48-40b6-a0d7-1b8c8bfbca37 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:34.763584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5e1e0afa-cd48-40b6-a0d7-1b8c8bfbca37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:34.772029 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1394/2774] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:20:34 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:34.773311 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d8233-9d20-4083-94b8-9bf3c2e2b956 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:34.776453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e0afa-cd48-40b6-a0d7-1b8c8bfbca37 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:34.838203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fe9bd-462d-4b9b-902c-4cdb8f67915d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:34.839357 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fe9bd-462d-4b9b-902c-4cdb8f67915d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:34.839641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fe9bd-462d-4b9b-902c-4cdb8f67915d admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:34.839918 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fe9bd-462d-4b9b-902c-4cdb8f67915d admin admin] PATH_INFO: `/v3/users/bbdcb7eea95b48a8a00327763653a21f` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:34.841224 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fe9bd-462d-4b9b-902c-4cdb8f67915d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbdcb7eea95b48a8a00327763653a21f)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:34.843561 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fe9bd-462d-4b9b-902c-4cdb8f67915d admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:34.897389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-9e5fe9bd-462d-4b9b-902c-4cdb8f67915d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbdcb7eea95b48a8a00327763653a21f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:34.909676 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-9e5fe9bd-462d-4b9b-902c-4cdb8f67915d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbdcb7eea95b48a8a00327763653a21f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:34.914892 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:INSERT=1,keystone:DELETE=10 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:34.917267 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-9e5fe9bd-462d-4b9b-902c-4cdb8f67915d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbdcb7eea95b48a8a00327763653a21f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:34.918208 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-9e5fe9bd-462d-4b9b-902c-4cdb8f67915d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbdcb7eea95b48a8a00327763653a21f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:34.928476 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-9e5fe9bd-462d-4b9b-902c-4cdb8f67915d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbdcb7eea95b48a8a00327763653a21f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:34.934826 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1381/2775] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:20:34 2022] DELETE /identity/v3/users/bbdcb7eea95b48a8a00327763653a21f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:20:34.939700 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194cefd-c95e-4849-9981-b82bd7497fbc None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:34.940048 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194cefd-c95e-4849-9981-b82bd7497fbc None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:34.940336 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194cefd-c95e-4849-9981-b82bd7497fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:34.957609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-4194cefd-c95e-4849-9981-b82bd7497fbc None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:34.958834 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4194cefd-c95e-4849-9981-b82bd7497fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:34.963897 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d8233-9d20-4083-94b8-9bf3c2e2b956 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:35.018486 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1382/2776] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:20:35.020888 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d8372-92c1-426e-aa4d-3a4da70d0d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:35.021564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d8372-92c1-426e-aa4d-3a4da70d0d72 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:35.021849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d8372-92c1-426e-aa4d-3a4da70d0d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:35.022130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d8372-92c1-426e-aa4d-3a4da70d0d72 admin admin] PATH_INFO: `/v3/users` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:35.022846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d8372-92c1-426e-aa4d-3a4da70d0d72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:35.024738 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d8372-92c1-426e-aa4d-3a4da70d0d72 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:35.028736 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194cefd-c95e-4849-9981-b82bd7497fbc None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:35.030842 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4194cefd-c95e-4849-9981-b82bd7497fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:35.041149 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:35.048027 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1395/2777] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:20:34 2022] POST /identity/v3/users => generated 294 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:35.051667 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d8372-92c1-426e-aa4d-3a4da70d0d72 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:35.053371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e36d8372-92c1-426e-aa4d-3a4da70d0d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:35.064959 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d8372-92c1-426e-aa4d-3a4da70d0d72 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:35.069504 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c139e-a3ce-43ca-906a-ce64e361b658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:35.070155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c139e-a3ce-43ca-906a-ce64e361b658 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:35.070606 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c139e-a3ce-43ca-906a-ce64e361b658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:35.070910 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c139e-a3ce-43ca-906a-ce64e361b658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:35.071634 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c139e-a3ce-43ca-906a-ce64e361b658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:35.073197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c139e-a3ce-43ca-906a-ce64e361b658 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:35.083764 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1396/2778] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:20:35.087690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c139e-a3ce-43ca-906a-ce64e361b658 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:35.095443 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194cefd-c95e-4849-9981-b82bd7497fbc None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:35.098178 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c139e-a3ce-43ca-906a-ce64e361b658 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:35.100370 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10115294-539d-491d-9c88-481123c96f3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:35.101112 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10115294-539d-491d-9c88-481123c96f3c service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:35.101430 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10115294-539d-491d-9c88-481123c96f3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:35.101853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10115294-539d-491d-9c88-481123c96f3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:35.102976 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c30f0-20c1-4221-b55a-e4d03eb2ad03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:35.103666 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c30f0-20c1-4221-b55a-e4d03eb2ad03 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:35.103957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c30f0-20c1-4221-b55a-e4d03eb2ad03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:35.104233 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c30f0-20c1-4221-b55a-e4d03eb2ad03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:35.104957 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c30f0-20c1-4221-b55a-e4d03eb2ad03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:35.106606 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c30f0-20c1-4221-b55a-e4d03eb2ad03 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:35.117013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10115294-539d-491d-9c88-481123c96f3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:35.118180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1397/2779] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:20:35.119811 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10115294-539d-491d-9c88-481123c96f3c service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:35.124478 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c30f0-20c1-4221-b55a-e4d03eb2ad03 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:35.133208 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1383/2780] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:20:35.134558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c30f0-20c1-4221-b55a-e4d03eb2ad03 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:35.140944 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c0562-0892-4f00-b1c1-66d55e395edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:35.141846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c0562-0892-4f00-b1c1-66d55e395edd admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:35.142781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c0562-0892-4f00-b1c1-66d55e395edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:35.143292 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c0562-0892-4f00-b1c1-66d55e395edd admin admin] PATH_INFO: `/v3/projects/f00ce04db9344623bfb40690490f0c05/users/d726b9a09e644fe0b7372e5639b20df7/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:35.160693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c0562-0892-4f00-b1c1-66d55e395edd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00ce04db9344623bfb40690490f0c05, user_id=d726b9a09e644fe0b7372e5639b20df7, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:35.164715 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40c0562-0892-4f00-b1c1-66d55e395edd admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:35.183542 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1398/2781] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:35 2022] PUT /identity/v3/projects/f00ce04db9344623bfb40690490f0c05/users/d726b9a09e644fe0b7372e5639b20df7/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:20:35.195353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399be2f-69ea-4ab0-9c5c-679e8335e33a None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:35.195643 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399be2f-69ea-4ab0-9c5c-679e8335e33a None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:35.195927 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399be2f-69ea-4ab0-9c5c-679e8335e33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:35.234391 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-e399be2f-69ea-4ab0-9c5c-679e8335e33a None None] MFA Rules not processed for user `d726b9a09e644fe0b7372e5639b20df7`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:35.235746 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e399be2f-69ea-4ab0-9c5c-679e8335e33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:35.311792 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1384/2782] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:20:35.316294 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c0562-0892-4f00-b1c1-66d55e395edd admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:35.382170 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40c0562-0892-4f00-b1c1-66d55e395edd admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:35.387471 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb5d56-3a52-4bb5-9560-a4799e2f6f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:35.388152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb5d56-3a52-4bb5-9560-a4799e2f6f08 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:35.388439 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb5d56-3a52-4bb5-9560-a4799e2f6f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:35.388822 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb5d56-3a52-4bb5-9560-a4799e2f6f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:35.389965 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb5d56-3a52-4bb5-9560-a4799e2f6f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:35.391792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb5d56-3a52-4bb5-9560-a4799e2f6f08 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:35.429637 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1399/2783] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:20:35 2022] POST /identity/v3/projects => generated 389 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:35.434115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399be2f-69ea-4ab0-9c5c-679e8335e33a None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:35.445566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399be2f-69ea-4ab0-9c5c-679e8335e33a None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:35.464970 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:35.484493 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb5d56-3a52-4bb5-9560-a4799e2f6f08 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:35.486312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-44bb5d56-3a52-4bb5-9560-a4799e2f6f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:35.513472 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5f23d-90a1-497c-85d7-3f596ef5313c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:35.514479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5f23d-90a1-497c-85d7-3f596ef5313c admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:35.515391 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5f23d-90a1-497c-85d7-3f596ef5313c admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:35.515840 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5f23d-90a1-497c-85d7-3f596ef5313c admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:35.516990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d5f23d-90a1-497c-85d7-3f596ef5313c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:35.518961 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d5f23d-90a1-497c-85d7-3f596ef5313c admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:35.541756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1385/2784] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:20:35 2022] POST /identity/v3/users => generated 301 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:20:35.547057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5f23d-90a1-497c-85d7-3f596ef5313c admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:35.562134 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb5d56-3a52-4bb5-9560-a4799e2f6f08 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:35.562348 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5f23d-90a1-497c-85d7-3f596ef5313c admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:35.566850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c334e73-963e-4328-93cf-2140e69bf7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:35.567228 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb58c7-1716-4594-b360-54f62ec9355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:35.567717 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c334e73-963e-4328-93cf-2140e69bf7a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:35.567882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb58c7-1716-4594-b360-54f62ec9355f admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:35.568002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c334e73-963e-4328-93cf-2140e69bf7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:35.568163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb58c7-1716-4594-b360-54f62ec9355f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:35.568266 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c334e73-963e-4328-93cf-2140e69bf7a2 admin admin] PATH_INFO: `/v3/projects/5d4d6988adeb4b32af53259fdd5dbf94` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:35.568442 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb58c7-1716-4594-b360-54f62ec9355f admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:35.569189 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb58c7-1716-4594-b360-54f62ec9355f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:35.569449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c334e73-963e-4328-93cf-2140e69bf7a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d4d6988adeb4b32af53259fdd5dbf94)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:35.571020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb58c7-1716-4594-b360-54f62ec9355f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:35.571435 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c334e73-963e-4328-93cf-2140e69bf7a2 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:35.582092 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1386/2785] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:20:35.586098 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb58c7-1716-4594-b360-54f62ec9355f admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:35.595941 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb58c7-1716-4594-b360-54f62ec9355f admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:35.600658 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78476b-580d-4c15-83d7-c68682e538b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:35.601698 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78476b-580d-4c15-83d7-c68682e538b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:35.602144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78476b-580d-4c15-83d7-c68682e538b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:35.602496 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78476b-580d-4c15-83d7-c68682e538b3 admin admin] PATH_INFO: `/v3/projects/627f014e8abe49a6985ec5d4c09d9b1b/users/9061d74c8cde44b39504699cdedf0730/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:35.616728 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:35.620115 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c78476b-580d-4c15-83d7-c68682e538b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=627f014e8abe49a6985ec5d4c09d9b1b, user_id=9061d74c8cde44b39504699cdedf0730, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:35.622948 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c78476b-580d-4c15-83d7-c68682e538b3 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:35.624561 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7c334e73-963e-4328-93cf-2140e69bf7a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d4d6988adeb4b32af53259fdd5dbf94', 'request_id': 'req-7c334e73-963e-4328-93cf-2140e69bf7a2'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:35.631816 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7c334e73-963e-4328-93cf-2140e69bf7a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d4d6988adeb4b32af53259fdd5dbf94', 'request_id': 'req-7c334e73-963e-4328-93cf-2140e69bf7a2'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:35.637246 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1387/2786] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:35 2022] PUT /identity/v3/projects/627f014e8abe49a6985ec5d4c09d9b1b/users/9061d74c8cde44b39504699cdedf0730/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:20:35.641274 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78476b-580d-4c15-83d7-c68682e538b3 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:35.641418 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7c334e73-963e-4328-93cf-2140e69bf7a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d4d6988adeb4b32af53259fdd5dbf94', 'request_id': 'req-7c334e73-963e-4328-93cf-2140e69bf7a2'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:35.654042 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7c334e73-963e-4328-93cf-2140e69bf7a2 admin admin] The token cache is being invalidate because project 5d4d6988adeb4b32af53259fdd5dbf94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:35.654249 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7c334e73-963e-4328-93cf-2140e69bf7a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:35.655114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7c334e73-963e-4328-93cf-2140e69bf7a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:35.656689 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1400/2787] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:35 2022] DELETE /identity/v3/projects/5d4d6988adeb4b32af53259fdd5dbf94 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:20:35.712249 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78476b-580d-4c15-83d7-c68682e538b3 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:35.718145 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176e53c-a1e3-4582-8a02-0fce11e5f9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:35.718490 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176e53c-a1e3-4582-8a02-0fce11e5f9fc admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:35.718772 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176e53c-a1e3-4582-8a02-0fce11e5f9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:35.719051 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176e53c-a1e3-4582-8a02-0fce11e5f9fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:35.719796 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176e53c-a1e3-4582-8a02-0fce11e5f9fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:35.721437 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176e53c-a1e3-4582-8a02-0fce11e5f9fc admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:35.733883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1388/2788] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:35.738198 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c334e73-963e-4328-93cf-2140e69bf7a2 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:35.739749 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7c334e73-963e-4328-93cf-2140e69bf7a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:35.814278 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c334e73-963e-4328-93cf-2140e69bf7a2 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:35.819139 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd888268-14b4-4fc0-a0cb-5fdd1897d57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:35.820052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd888268-14b4-4fc0-a0cb-5fdd1897d57a admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:35.820349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd888268-14b4-4fc0-a0cb-5fdd1897d57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:35.820610 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd888268-14b4-4fc0-a0cb-5fdd1897d57a admin admin] PATH_INFO: `/v3/projects/627f014e8abe49a6985ec5d4c09d9b1b/users/9061d74c8cde44b39504699cdedf0730/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:35.822609 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd888268-14b4-4fc0-a0cb-5fdd1897d57a admin admin] RBAC: Authorizing `identity:create_grant(project_id=627f014e8abe49a6985ec5d4c09d9b1b, user_id=9061d74c8cde44b39504699cdedf0730, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:35.825531 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd888268-14b4-4fc0-a0cb-5fdd1897d57a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:35.836909 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1401/2789] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:35 2022] PUT /identity/v3/projects/627f014e8abe49a6985ec5d4c09d9b1b/users/9061d74c8cde44b39504699cdedf0730/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:35.848408 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffa0cf-45f8-4901-97ae-0e9207c51a88 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:35.848717 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffa0cf-45f8-4901-97ae-0e9207c51a88 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:35.848998 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffa0cf-45f8-4901-97ae-0e9207c51a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:35.877977 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a9ffa0cf-45f8-4901-97ae-0e9207c51a88 None None] MFA Rules not processed for user `9061d74c8cde44b39504699cdedf0730`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:35.879254 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a9ffa0cf-45f8-4901-97ae-0e9207c51a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:35.944784 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:35.951898 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1389/2790] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:35.961844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd888268-14b4-4fc0-a0cb-5fdd1897d57a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:35.963602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-fd888268-14b4-4fc0-a0cb-5fdd1897d57a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:36.023756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd888268-14b4-4fc0-a0cb-5fdd1897d57a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:36.027338 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65c46b-cc35-46d8-a5f4-ea374aadff4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99dc3db93e9045b8b3ef59a030822972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:36.027973 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65c46b-cc35-46d8-a5f4-ea374aadff4d service cinder] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:36.029065 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65c46b-cc35-46d8-a5f4-ea374aadff4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:36.029883 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65c46b-cc35-46d8-a5f4-ea374aadff4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:36.039855 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65c46b-cc35-46d8-a5f4-ea374aadff4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:36.041290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65c46b-cc35-46d8-a5f4-ea374aadff4d service cinder] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:36.054654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1402/2791] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:36.184478 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffa0cf-45f8-4901-97ae-0e9207c51a88 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:36.291903 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffa0cf-45f8-4901-97ae-0e9207c51a88 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:36.296579 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67753ac4-98ae-4ce2-8065-51730c1c1727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:36.297450 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67753ac4-98ae-4ce2-8065-51730c1c1727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:36.297733 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67753ac4-98ae-4ce2-8065-51730c1c1727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:36.298009 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67753ac4-98ae-4ce2-8065-51730c1c1727 admin admin] PATH_INFO: `/v3/users/d726b9a09e644fe0b7372e5639b20df7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:36.299292 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67753ac4-98ae-4ce2-8065-51730c1c1727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d726b9a09e644fe0b7372e5639b20df7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:36.301200 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67753ac4-98ae-4ce2-8065-51730c1c1727 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:36.348389 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-67753ac4-98ae-4ce2-8065-51730c1c1727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd726b9a09e644fe0b7372e5639b20df7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:36.360379 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-67753ac4-98ae-4ce2-8065-51730c1c1727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd726b9a09e644fe0b7372e5639b20df7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:36.367376 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-67753ac4-98ae-4ce2-8065-51730c1c1727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd726b9a09e644fe0b7372e5639b20df7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:36.368061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-67753ac4-98ae-4ce2-8065-51730c1c1727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd726b9a09e644fe0b7372e5639b20df7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:36.375502 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-67753ac4-98ae-4ce2-8065-51730c1c1727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd726b9a09e644fe0b7372e5639b20df7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:36.381614 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1390/2792] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:20:36 2022] DELETE /identity/v3/users/d726b9a09e644fe0b7372e5639b20df7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:36.386400 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d48bd9-b4a2-4ce7-a20a-d8f7a00c7277 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:36.386900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d48bd9-b4a2-4ce7-a20a-d8f7a00c7277 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:36.386997 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d48bd9-b4a2-4ce7-a20a-d8f7a00c7277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:36.434003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-75d48bd9-b4a2-4ce7-a20a-d8f7a00c7277 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:36.435461 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-75d48bd9-b4a2-4ce7-a20a-d8f7a00c7277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:36.475221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:36.512849 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1403/2793] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:36.526924 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67753ac4-98ae-4ce2-8065-51730c1c1727 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:36.529024 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-67753ac4-98ae-4ce2-8065-51730c1c1727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:36.585110 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:36.622221 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67753ac4-98ae-4ce2-8065-51730c1c1727 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:36.626669 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d87be-7fac-4959-9458-bb833385532b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:36.627351 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d87be-7fac-4959-9458-bb833385532b service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:36.627710 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d87be-7fac-4959-9458-bb833385532b service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:36.627993 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d87be-7fac-4959-9458-bb833385532b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:36.637167 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d87be-7fac-4959-9458-bb833385532b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:36.638600 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d87be-7fac-4959-9458-bb833385532b service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:36.648847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1391/2794] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:36.911316 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d48bd9-b4a2-4ce7-a20a-d8f7a00c7277 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:36.912767 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-75d48bd9-b4a2-4ce7-a20a-d8f7a00c7277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:36.926026 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d48bd9-b4a2-4ce7-a20a-d8f7a00c7277 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:36.930906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457a6b3-b4ba-4453-b3aa-121455d4d332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:36.931931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457a6b3-b4ba-4453-b3aa-121455d4d332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:36.932193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457a6b3-b4ba-4453-b3aa-121455d4d332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:36.932543 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457a6b3-b4ba-4453-b3aa-121455d4d332 admin admin] PATH_INFO: `/v3/projects/f00ce04db9344623bfb40690490f0c05` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:36.933820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457a6b3-b4ba-4453-b3aa-121455d4d332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f00ce04db9344623bfb40690490f0c05)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:36.937034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457a6b3-b4ba-4453-b3aa-121455d4d332 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:36.986286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7457a6b3-b4ba-4453-b3aa-121455d4d332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f00ce04db9344623bfb40690490f0c05', 'request_id': 'req-7457a6b3-b4ba-4453-b3aa-121455d4d332'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:36.992006 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7457a6b3-b4ba-4453-b3aa-121455d4d332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f00ce04db9344623bfb40690490f0c05', 'request_id': 'req-7457a6b3-b4ba-4453-b3aa-121455d4d332'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:36.998669 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7457a6b3-b4ba-4453-b3aa-121455d4d332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f00ce04db9344623bfb40690490f0c05', 'request_id': 'req-7457a6b3-b4ba-4453-b3aa-121455d4d332'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:37.010599 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7457a6b3-b4ba-4453-b3aa-121455d4d332 admin admin] The token cache is being invalidate because project f00ce04db9344623bfb40690490f0c05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:37.010918 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7457a6b3-b4ba-4453-b3aa-121455d4d332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:37.011560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7457a6b3-b4ba-4453-b3aa-121455d4d332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:37.013089 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1404/2795] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:36 2022] DELETE /identity/v3/projects/f00ce04db9344623bfb40690490f0c05 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:37.017244 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d87be-7fac-4959-9458-bb833385532b service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:37.026862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d87be-7fac-4959-9458-bb833385532b service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:37.083318 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ee25a-b01c-428e-9e7f-e759a95c5b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:37.085009 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ee25a-b01c-428e-9e7f-e759a95c5b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:37.085494 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ee25a-b01c-428e-9e7f-e759a95c5b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:37.085985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ee25a-b01c-428e-9e7f-e759a95c5b6d admin admin] PATH_INFO: `/v3/users/9061d74c8cde44b39504699cdedf0730` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:37.088119 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377ee25a-b01c-428e-9e7f-e759a95c5b6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9061d74c8cde44b39504699cdedf0730)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:37.091958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377ee25a-b01c-428e-9e7f-e759a95c5b6d admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:37.140055 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-377ee25a-b01c-428e-9e7f-e759a95c5b6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9061d74c8cde44b39504699cdedf0730'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:37.151743 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-377ee25a-b01c-428e-9e7f-e759a95c5b6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9061d74c8cde44b39504699cdedf0730'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:37.158786 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-377ee25a-b01c-428e-9e7f-e759a95c5b6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9061d74c8cde44b39504699cdedf0730'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:37.159448 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-377ee25a-b01c-428e-9e7f-e759a95c5b6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9061d74c8cde44b39504699cdedf0730'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:37.166998 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-377ee25a-b01c-428e-9e7f-e759a95c5b6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9061d74c8cde44b39504699cdedf0730'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:37.174187 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1392/2796] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:20:37 2022] DELETE /identity/v3/users/9061d74c8cde44b39504699cdedf0730 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:37.443951 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457a6b3-b4ba-4453-b3aa-121455d4d332 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:37.445451 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7457a6b3-b4ba-4453-b3aa-121455d4d332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:37.460671 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457a6b3-b4ba-4453-b3aa-121455d4d332 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:37.517889 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aaded1-068a-4f3e-9c3b-9c916a8eda82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:37.518789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaded1-068a-4f3e-9c3b-9c916a8eda82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:37.519101 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaded1-068a-4f3e-9c3b-9c916a8eda82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:37.519524 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aaded1-068a-4f3e-9c3b-9c916a8eda82 admin admin] PATH_INFO: `/v3/projects/627f014e8abe49a6985ec5d4c09d9b1b` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:37.520698 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aaded1-068a-4f3e-9c3b-9c916a8eda82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=627f014e8abe49a6985ec5d4c09d9b1b)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:37.522639 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aaded1-068a-4f3e-9c3b-9c916a8eda82 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:37.569974 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-a9aaded1-068a-4f3e-9c3b-9c916a8eda82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '627f014e8abe49a6985ec5d4c09d9b1b', 'request_id': 'req-a9aaded1-068a-4f3e-9c3b-9c916a8eda82'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:37.575820 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-a9aaded1-068a-4f3e-9c3b-9c916a8eda82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '627f014e8abe49a6985ec5d4c09d9b1b', 'request_id': 'req-a9aaded1-068a-4f3e-9c3b-9c916a8eda82'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:37.582498 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-a9aaded1-068a-4f3e-9c3b-9c916a8eda82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '627f014e8abe49a6985ec5d4c09d9b1b', 'request_id': 'req-a9aaded1-068a-4f3e-9c3b-9c916a8eda82'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:37.585030 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:37.596152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-a9aaded1-068a-4f3e-9c3b-9c916a8eda82 admin admin] The token cache is being invalidate because project 627f014e8abe49a6985ec5d4c09d9b1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:20:37.596437 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-a9aaded1-068a-4f3e-9c3b-9c916a8eda82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:37.597054 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-a9aaded1-068a-4f3e-9c3b-9c916a8eda82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:20:37.598596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1405/2797] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:20:37 2022] DELETE /identity/v3/projects/627f014e8abe49a6985ec5d4c09d9b1b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:37.603601 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f7599-6661-49b9-8217-2635a3ecb823 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:37.604068 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f7599-6661-49b9-8217-2635a3ecb823 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:37.604353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f7599-6661-49b9-8217-2635a3ecb823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:37.651204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-9f2f7599-6661-49b9-8217-2635a3ecb823 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:37.652414 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-9f2f7599-6661-49b9-8217-2635a3ecb823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:37.666632 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:37.722934 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1393/2798] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:37.732756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e7a40-f5d7-4d93-9396-d9ec4508f9c2 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:37.733224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e7a40-f5d7-4d93-9396-d9ec4508f9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:37.733506 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e7a40-f5d7-4d93-9396-d9ec4508f9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:37.752842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-e09e7a40-f5d7-4d93-9396-d9ec4508f9c2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:37.754085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e09e7a40-f5d7-4d93-9396-d9ec4508f9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:37.762194 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1406/2799] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:37.767202 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f7599-6661-49b9-8217-2635a3ecb823 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:37.777379 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f7599-6661-49b9-8217-2635a3ecb823 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:37.783099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09637072-9c56-4efb-ac29-b95b74563cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:37.783812 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09637072-9c56-4efb-ac29-b95b74563cdb admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:37.784099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09637072-9c56-4efb-ac29-b95b74563cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:37.784448 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09637072-9c56-4efb-ac29-b95b74563cdb admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:37.785151 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09637072-9c56-4efb-ac29-b95b74563cdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:20:37.785450 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09637072-9c56-4efb-ac29-b95b74563cdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:37.787181 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09637072-9c56-4efb-ac29-b95b74563cdb admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:37.799879 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1394/2800] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:20:37.804414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e7a40-f5d7-4d93-9396-d9ec4508f9c2 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:37.814105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e7a40-f5d7-4d93-9396-d9ec4508f9c2 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:37.818713 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7c84a-37c8-4c13-9c73-93378002ffc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:37.819376 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7c84a-37c8-4c13-9c73-93378002ffc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:37.819660 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7c84a-37c8-4c13-9c73-93378002ffc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:37.819936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7c84a-37c8-4c13-9c73-93378002ffc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:37.820624 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7c84a-37c8-4c13-9c73-93378002ffc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:37.822538 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7c84a-37c8-4c13-9c73-93378002ffc8 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:37.850122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1407/2801] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:20:37 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:20:37.854282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09637072-9c56-4efb-ac29-b95b74563cdb admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:37.864670 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09637072-9c56-4efb-ac29-b95b74563cdb admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:37.923705 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73714fa5-b76a-4664-b108-07fc29515252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:37.924391 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73714fa5-b76a-4664-b108-07fc29515252 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:37.924675 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73714fa5-b76a-4664-b108-07fc29515252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:37.924999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73714fa5-b76a-4664-b108-07fc29515252 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:37.925642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73714fa5-b76a-4664-b108-07fc29515252 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:37.927597 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73714fa5-b76a-4664-b108-07fc29515252 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:37.949636 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1395/2802] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:20:37 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:20:37.952967 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7c84a-37c8-4c13-9c73-93378002ffc8 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:37.963328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7c84a-37c8-4c13-9c73-93378002ffc8 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:37.968594 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2328d1-f38b-4eca-b495-b8e298186453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:37.969275 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2328d1-f38b-4eca-b495-b8e298186453 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:37.969570 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2328d1-f38b-4eca-b495-b8e298186453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:37.969852 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2328d1-f38b-4eca-b495-b8e298186453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:37.970582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2328d1-f38b-4eca-b495-b8e298186453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:37.972187 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2328d1-f38b-4eca-b495-b8e298186453 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:37.982723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1408/2803] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:20:37.986614 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73714fa5-b76a-4664-b108-07fc29515252 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:37.996685 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73714fa5-b76a-4664-b108-07fc29515252 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:38.001911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703963c8-5b05-4f1a-9136-dcdf3f081f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:38.002569 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703963c8-5b05-4f1a-9136-dcdf3f081f39 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:38.002884 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703963c8-5b05-4f1a-9136-dcdf3f081f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:38.003164 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703963c8-5b05-4f1a-9136-dcdf3f081f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:38.003882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703963c8-5b05-4f1a-9136-dcdf3f081f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:38.005445 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703963c8-5b05-4f1a-9136-dcdf3f081f39 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:38.015758 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1396/2804] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:38.019688 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2328d1-f38b-4eca-b495-b8e298186453 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:38.029614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2328d1-f38b-4eca-b495-b8e298186453 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:38.034343 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5586e2-3824-4ad1-8406-26ab8fc58de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:38.035265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5586e2-3824-4ad1-8406-26ab8fc58de1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:38.035557 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5586e2-3824-4ad1-8406-26ab8fc58de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:38.035834 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5586e2-3824-4ad1-8406-26ab8fc58de1 admin admin] PATH_INFO: `/v3/projects/96007c68c7f349a19f6bc8c16f024cc6/users/5a2f15a789f1482ba7a4ecbd260913ea/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:38.050900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5586e2-3824-4ad1-8406-26ab8fc58de1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96007c68c7f349a19f6bc8c16f024cc6, user_id=5a2f15a789f1482ba7a4ecbd260913ea, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:38.053523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5586e2-3824-4ad1-8406-26ab8fc58de1 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:38.068479 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1409/2805] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:38 2022] PUT /identity/v3/projects/96007c68c7f349a19f6bc8c16f024cc6/users/5a2f15a789f1482ba7a4ecbd260913ea/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:38.073540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cae9dc-bc6b-4bfa-a9ec-21b3898102a1 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:38.073837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cae9dc-bc6b-4bfa-a9ec-21b3898102a1 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:38.074125 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cae9dc-bc6b-4bfa-a9ec-21b3898102a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:38.094031 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-49cae9dc-bc6b-4bfa-a9ec-21b3898102a1 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:38.095274 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-49cae9dc-bc6b-4bfa-a9ec-21b3898102a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:38.160408 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1397/2806] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:38.170182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5586e2-3824-4ad1-8406-26ab8fc58de1 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:38.171963 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-1d5586e2-3824-4ad1-8406-26ab8fc58de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:38.237815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5586e2-3824-4ad1-8406-26ab8fc58de1 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:38.241340 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed999aa0-3aa1-4717-858d-f9077e2aacb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:38.242021 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed999aa0-3aa1-4717-858d-f9077e2aacb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:38.242308 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed999aa0-3aa1-4717-858d-f9077e2aacb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:38.242621 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed999aa0-3aa1-4717-858d-f9077e2aacb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:38.253144 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed999aa0-3aa1-4717-858d-f9077e2aacb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:38.255296 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed999aa0-3aa1-4717-858d-f9077e2aacb6 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:38.265290 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1410/2807] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:20:42.581841 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8574445-618a-4848-b470-86ca7f719897 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:42.582383 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8574445-618a-4848-b470-86ca7f719897 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:42.582487 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8574445-618a-4848-b470-86ca7f719897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:42.621152 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-a8574445-618a-4848-b470-86ca7f719897 None None] MFA Rules not processed for user `5a2f15a789f1482ba7a4ecbd260913ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:42.622300 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a8574445-618a-4848-b470-86ca7f719897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:42.640268 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:42.703667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1398/2808] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:42.707881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed999aa0-3aa1-4717-858d-f9077e2aacb6 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:42.718141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed999aa0-3aa1-4717-858d-f9077e2aacb6 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:42.723025 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db9949-63fd-45af-8c11-6681d749fae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:42.724029 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db9949-63fd-45af-8c11-6681d749fae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:42.724400 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db9949-63fd-45af-8c11-6681d749fae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:42.724748 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db9949-63fd-45af-8c11-6681d749fae1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:42.725524 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db9949-63fd-45af-8c11-6681d749fae1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:42.727596 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db9949-63fd-45af-8c11-6681d749fae1 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:42.772949 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1411/2809] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:20:42 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:20:42.776714 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8574445-618a-4848-b470-86ca7f719897 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:42.791551 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8574445-618a-4848-b470-86ca7f719897 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:42.855693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623df8f-58f4-4888-97ee-068ca511e564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:42.856387 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623df8f-58f4-4888-97ee-068ca511e564 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:42.856674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623df8f-58f4-4888-97ee-068ca511e564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:42.856958 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623df8f-58f4-4888-97ee-068ca511e564 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:42.858378 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623df8f-58f4-4888-97ee-068ca511e564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:42.860596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623df8f-58f4-4888-97ee-068ca511e564 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:42.883816 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1399/2810] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:20:42 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:20:42.887523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db9949-63fd-45af-8c11-6681d749fae1 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:42.902446 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db9949-63fd-45af-8c11-6681d749fae1 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:42.910768 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e5148-0755-4d73-8a35-d85c04ee69d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:42.911565 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e5148-0755-4d73-8a35-d85c04ee69d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:42.911931 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e5148-0755-4d73-8a35-d85c04ee69d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:42.912554 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e5148-0755-4d73-8a35-d85c04ee69d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:42.913194 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e5148-0755-4d73-8a35-d85c04ee69d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:42.916232 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e5148-0755-4d73-8a35-d85c04ee69d9 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:42.929781 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1412/2811] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:42.933794 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623df8f-58f4-4888-97ee-068ca511e564 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:42.946479 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623df8f-58f4-4888-97ee-068ca511e564 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:42.952562 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042612b3-aad3-46c5-938a-79632e6d4834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:42.953614 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042612b3-aad3-46c5-938a-79632e6d4834 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:42.953829 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042612b3-aad3-46c5-938a-79632e6d4834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:42.954113 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042612b3-aad3-46c5-938a-79632e6d4834 admin admin] PATH_INFO: `/v3/projects/263a1059e7af42ed9faaa331a3b4c76a/users/50aeb5eaa5b745aeb8e937a27584b606/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:42.971466 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042612b3-aad3-46c5-938a-79632e6d4834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=263a1059e7af42ed9faaa331a3b4c76a, user_id=50aeb5eaa5b745aeb8e937a27584b606, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:42.974040 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042612b3-aad3-46c5-938a-79632e6d4834 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:42.991619 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1400/2812] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:42 2022] PUT /identity/v3/projects/263a1059e7af42ed9faaa331a3b4c76a/users/50aeb5eaa5b745aeb8e937a27584b606/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:20:42.995497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e5148-0755-4d73-8a35-d85c04ee69d9 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:43.005999 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e5148-0755-4d73-8a35-d85c04ee69d9 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:43.071155 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2f641-c5a7-4fd8-8144-f4a12073ee9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:43.071953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2f641-c5a7-4fd8-8144-f4a12073ee9d admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:43.072300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2f641-c5a7-4fd8-8144-f4a12073ee9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:43.072642 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2f641-c5a7-4fd8-8144-f4a12073ee9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:43.073436 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2f641-c5a7-4fd8-8144-f4a12073ee9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:43.075371 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2f641-c5a7-4fd8-8144-f4a12073ee9d admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:43.077217 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:43.088392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1413/2813] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:20:43.093343 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042612b3-aad3-46c5-938a-79632e6d4834 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:43.109252 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042612b3-aad3-46c5-938a-79632e6d4834 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:43.117319 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790f411-98d9-4c7e-a3db-81f13e2470c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:43.118232 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790f411-98d9-4c7e-a3db-81f13e2470c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:43.118514 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790f411-98d9-4c7e-a3db-81f13e2470c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:43.118820 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790f411-98d9-4c7e-a3db-81f13e2470c7 admin admin] PATH_INFO: `/v3/projects/263a1059e7af42ed9faaa331a3b4c76a/users/50aeb5eaa5b745aeb8e937a27584b606/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:43.120964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790f411-98d9-4c7e-a3db-81f13e2470c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=263a1059e7af42ed9faaa331a3b4c76a, user_id=50aeb5eaa5b745aeb8e937a27584b606, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:43.125609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790f411-98d9-4c7e-a3db-81f13e2470c7 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:43.138381 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1401/2814] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:20:43 2022] PUT /identity/v3/projects/263a1059e7af42ed9faaa331a3b4c76a/users/50aeb5eaa5b745aeb8e937a27584b606/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:20:44.913117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3438b-ac77-4c8a-80dd-6745ea4f9c85 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:44.913698 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3438b-ac77-4c8a-80dd-6745ea4f9c85 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:44.913698 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3438b-ac77-4c8a-80dd-6745ea4f9c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:44.933590 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-3cd3438b-ac77-4c8a-80dd-6745ea4f9c85 None None] MFA Rules not processed for user `c6cd8828e7b140569aaea330051b00d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:44.934869 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-3cd3438b-ac77-4c8a-80dd-6745ea4f9c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:45.006044 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1414/2815] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:49.363256 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a5463-9ae2-432e-941e-d54e496af8c1 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:49.363832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a5463-9ae2-432e-941e-d54e496af8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:49.363832 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a5463-9ae2-432e-941e-d54e496af8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:49.397639 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-886a5463-9ae2-432e-941e-d54e496af8c1 None None] MFA Rules not processed for user `50aeb5eaa5b745aeb8e937a27584b606`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:49.398928 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-886a5463-9ae2-432e-941e-d54e496af8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:49.457891 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:49.478086 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1402/2816] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:49.490323 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3438b-ac77-4c8a-80dd-6745ea4f9c85 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:49.491750 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-3cd3438b-ac77-4c8a-80dd-6745ea4f9c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:49.575810 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3438b-ac77-4c8a-80dd-6745ea4f9c85 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:49.581333 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f98d37-1114-4497-b8f9-f7167a621e75 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb93eb1d25c4410591125b0ab7a73149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:49.582116 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f98d37-1114-4497-b8f9-f7167a621e75 service glance] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:49.582477 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f98d37-1114-4497-b8f9-f7167a621e75 service glance] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:49.582862 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f98d37-1114-4497-b8f9-f7167a621e75 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:49.653210 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f98d37-1114-4497-b8f9-f7167a621e75 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:49.655212 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f98d37-1114-4497-b8f9-f7167a621e75 service glance] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:49.665761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1415/2817] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:20:49.776252 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda6753-23f4-4791-beb1-7b903b4f703b None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:49.776543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda6753-23f4-4791-beb1-7b903b4f703b None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:49.776850 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda6753-23f4-4791-beb1-7b903b4f703b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:49.847863 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-4eda6753-23f4-4791-beb1-7b903b4f703b None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:49.849083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4eda6753-23f4-4791-beb1-7b903b4f703b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:49.936923 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1403/2818] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jul 20 09:20:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:50.004620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f98d37-1114-4497-b8f9-f7167a621e75 service glance] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:50.006259 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e9f98d37-1114-4497-b8f9-f7167a621e75 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:50.081274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=100 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:20:50.098070 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f98d37-1114-4497-b8f9-f7167a621e75 service glance] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:50.108331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d270665-3721-4da2-aafd-e7a6ee698337 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:50.109241 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d270665-3721-4da2-aafd-e7a6ee698337 service swift] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:50.109732 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d270665-3721-4da2-aafd-e7a6ee698337 service swift] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:50.110269 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d270665-3721-4da2-aafd-e7a6ee698337 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:50.120338 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d270665-3721-4da2-aafd-e7a6ee698337 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:50.122222 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d270665-3721-4da2-aafd-e7a6ee698337 service swift] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:50.131611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1416/2819] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:20:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:20:59.287272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70872d38-26ee-4bda-ae83-cdf1ffb4551d None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:59.287837 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70872d38-26ee-4bda-ae83-cdf1ffb4551d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:59.287998 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70872d38-26ee-4bda-ae83-cdf1ffb4551d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:59.306988 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-70872d38-26ee-4bda-ae83-cdf1ffb4551d None None] MFA Rules not processed for user `5a2f15a789f1482ba7a4ecbd260913ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:20:59.308201 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-70872d38-26ee-4bda-ae83-cdf1ffb4551d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:59.317527 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1404/2820] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:20:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:20:59.531496 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d270665-3721-4da2-aafd-e7a6ee698337 service swift] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:20:59.532955 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2d270665-3721-4da2-aafd-e7a6ee698337 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:59.637152 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d270665-3721-4da2-aafd-e7a6ee698337 service swift] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:20:59.642672 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a595b-bd7d-44e7-a1ad-28ac37671136 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '066f69a87f5f48c7b20c1e0fcf5c1107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:20:59.643347 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a595b-bd7d-44e7-a1ad-28ac37671136 service placement] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:20:59.643640 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a595b-bd7d-44e7-a1ad-28ac37671136 service placement] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:20:59.643920 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a595b-bd7d-44e7-a1ad-28ac37671136 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:20:59.645719 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ca9a595b-bd7d-44e7-a1ad-28ac37671136 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:20:59.658121 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a595b-bd7d-44e7-a1ad-28ac37671136 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:20:59.659760 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a595b-bd7d-44e7-a1ad-28ac37671136 service placement] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:20:59.670348 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1417/2821] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:20:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:21:01.010251 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a12323-d86f-4f68-8e49-a22acc6976d3 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:01.010826 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a12323-d86f-4f68-8e49-a22acc6976d3 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:01.010826 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a12323-d86f-4f68-8e49-a22acc6976d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:01.032804 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-86a12323-d86f-4f68-8e49-a22acc6976d3 None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:01.034756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-86a12323-d86f-4f68-8e49-a22acc6976d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:01.043547 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1405/2822] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jul 20 09:21:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:21:01.065451 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a595b-bd7d-44e7-a1ad-28ac37671136 service placement] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:01.077875 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a595b-bd7d-44e7-a1ad-28ac37671136 service placement] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:01.086313 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97b4e8-428b-47a5-8a26-24c8a588c2e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:01.087003 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97b4e8-428b-47a5-8a26-24c8a588c2e7 service swift] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:01.087668 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97b4e8-428b-47a5-8a26-24c8a588c2e7 service swift] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:01.087668 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97b4e8-428b-47a5-8a26-24c8a588c2e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:01.098085 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97b4e8-428b-47a5-8a26-24c8a588c2e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:01.099628 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97b4e8-428b-47a5-8a26-24c8a588c2e7 service swift] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:01.108034 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1418/2823] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:21:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:21:06.336291 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f67dd5-5cb3-4df7-b8f3-9a30d7316f03 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:06.336894 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f67dd5-5cb3-4df7-b8f3-9a30d7316f03 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:06.337777 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f67dd5-5cb3-4df7-b8f3-9a30d7316f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:06.356611 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-c1f67dd5-5cb3-4df7-b8f3-9a30d7316f03 None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:06.357990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-c1f67dd5-5cb3-4df7-b8f3-9a30d7316f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:06.366497 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1406/2824] 10.222.0.44 () {58 vars in 1009 bytes} [Wed Jul 20 09:21:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 09:21:06.380396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97b4e8-428b-47a5-8a26-24c8a588c2e7 service swift] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:06.394723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97b4e8-428b-47a5-8a26-24c8a588c2e7 service swift] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:06.401143 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae04d01-fd25-4397-9576-72e8b0bb181c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:06.402024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae04d01-fd25-4397-9576-72e8b0bb181c service swift] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:06.402400 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae04d01-fd25-4397-9576-72e8b0bb181c service swift] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:06.402822 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae04d01-fd25-4397-9576-72e8b0bb181c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:06.413906 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae04d01-fd25-4397-9576-72e8b0bb181c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:06.415564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae04d01-fd25-4397-9576-72e8b0bb181c service swift] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:06.425201 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1419/2825] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:21:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:21:16.351203 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=45 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:16.421216 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=60 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:22.277196 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e9d95-7bd3-45fc-81d5-3516aac9370f None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:22.277732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e9d95-7bd3-45fc-81d5-3516aac9370f None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:22.277732 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e9d95-7bd3-45fc-81d5-3516aac9370f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:22.296763 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-bb6e9d95-7bd3-45fc-81d5-3516aac9370f None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:22.297954 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-bb6e9d95-7bd3-45fc-81d5-3516aac9370f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:22.305121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1407/2826] 10.222.0.44 () {58 vars in 967 bytes} [Wed Jul 20 09:21:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:21:22.317503 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae04d01-fd25-4397-9576-72e8b0bb181c service swift] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:22.328868 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae04d01-fd25-4397-9576-72e8b0bb181c service swift] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:22.334141 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597a7c1-0130-44f9-8590-c595f9d624fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:22.334977 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597a7c1-0130-44f9-8590-c595f9d624fd service swift] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:22.335270 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597a7c1-0130-44f9-8590-c595f9d624fd service swift] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:22.335558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597a7c1-0130-44f9-8590-c595f9d624fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:22.344927 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597a7c1-0130-44f9-8590-c595f9d624fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:22.346512 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597a7c1-0130-44f9-8590-c595f9d624fd service swift] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:22.355331 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1420/2827] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:21:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 09:21:24.250296 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e9d95-7bd3-45fc-81d5-3516aac9370f None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:24.251768 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-bb6e9d95-7bd3-45fc-81d5-3516aac9370f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:24.318520 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e9d95-7bd3-45fc-81d5-3516aac9370f None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:24.323009 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c53a7d-386f-4450-bbfc-ac057e9d2816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:24.323882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c53a7d-386f-4450-bbfc-ac057e9d2816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:24.324163 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c53a7d-386f-4450-bbfc-ac057e9d2816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:24.324438 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c53a7d-386f-4450-bbfc-ac057e9d2816 admin admin] PATH_INFO: `/v3/users/c6cd8828e7b140569aaea330051b00d6` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:24.326576 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c53a7d-386f-4450-bbfc-ac057e9d2816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6cd8828e7b140569aaea330051b00d6)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:24.328544 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c53a7d-386f-4450-bbfc-ac057e9d2816 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:24.386587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b9c53a7d-386f-4450-bbfc-ac057e9d2816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6cd8828e7b140569aaea330051b00d6'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:24.402882 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b9c53a7d-386f-4450-bbfc-ac057e9d2816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6cd8828e7b140569aaea330051b00d6'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:24.409971 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b9c53a7d-386f-4450-bbfc-ac057e9d2816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6cd8828e7b140569aaea330051b00d6'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:24.410779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b9c53a7d-386f-4450-bbfc-ac057e9d2816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6cd8828e7b140569aaea330051b00d6'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:24.424530 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-b9c53a7d-386f-4450-bbfc-ac057e9d2816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6cd8828e7b140569aaea330051b00d6'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:24.432951 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1408/2828] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:21:24 2022] DELETE /identity/v3/users/c6cd8828e7b140569aaea330051b00d6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:21:24.655175 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597a7c1-0130-44f9-8590-c595f9d624fd service swift] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:24.666784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597a7c1-0130-44f9-8590-c595f9d624fd service swift] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:24.731553 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3eb3a8-0769-4395-b199-e90f8ad024ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:24.732524 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3eb3a8-0769-4395-b199-e90f8ad024ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:24.732815 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3eb3a8-0769-4395-b199-e90f8ad024ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:24.733092 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3eb3a8-0769-4395-b199-e90f8ad024ab admin admin] PATH_INFO: `/v3/projects/89985f3bd3184d10a94b167827b39a3f` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:24.734293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3eb3a8-0769-4395-b199-e90f8ad024ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=89985f3bd3184d10a94b167827b39a3f)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:24.736226 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3eb3a8-0769-4395-b199-e90f8ad024ab admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:24.791779 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2c3eb3a8-0769-4395-b199-e90f8ad024ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89985f3bd3184d10a94b167827b39a3f', 'request_id': 'req-2c3eb3a8-0769-4395-b199-e90f8ad024ab'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:24.798663 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2c3eb3a8-0769-4395-b199-e90f8ad024ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89985f3bd3184d10a94b167827b39a3f', 'request_id': 'req-2c3eb3a8-0769-4395-b199-e90f8ad024ab'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:24.805643 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2c3eb3a8-0769-4395-b199-e90f8ad024ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89985f3bd3184d10a94b167827b39a3f', 'request_id': 'req-2c3eb3a8-0769-4395-b199-e90f8ad024ab'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:24.822356 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2c3eb3a8-0769-4395-b199-e90f8ad024ab admin admin] The token cache is being invalidate because project 89985f3bd3184d10a94b167827b39a3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:21:24.822650 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2c3eb3a8-0769-4395-b199-e90f8ad024ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:24.823312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2c3eb3a8-0769-4395-b199-e90f8ad024ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:24.824851 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1421/2829] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:21:24 2022] DELETE /identity/v3/projects/89985f3bd3184d10a94b167827b39a3f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:21:24.831082 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746cb50-144f-42f3-a836-39d034cb5e08 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:24.831608 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746cb50-144f-42f3-a836-39d034cb5e08 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:24.832124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746cb50-144f-42f3-a836-39d034cb5e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:24.856426 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-6746cb50-144f-42f3-a836-39d034cb5e08 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:24.857678 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-6746cb50-144f-42f3-a836-39d034cb5e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:24.935797 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1409/2830] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:21:24.943855 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07829e4e-deb5-4ed5-ba71-1e581237f007 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:24.944171 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07829e4e-deb5-4ed5-ba71-1e581237f007 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:24.944454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07829e4e-deb5-4ed5-ba71-1e581237f007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:24.965136 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-07829e4e-deb5-4ed5-ba71-1e581237f007 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:24.967286 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-07829e4e-deb5-4ed5-ba71-1e581237f007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:24.981893 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1422/2831] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:21:24.986485 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746cb50-144f-42f3-a836-39d034cb5e08 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:24.997102 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746cb50-144f-42f3-a836-39d034cb5e08 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:25.003472 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb6acf-9eb6-4d1b-8dbb-d2acbb5b08f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:25.004138 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb6acf-9eb6-4d1b-8dbb-d2acbb5b08f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:25.004418 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb6acf-9eb6-4d1b-8dbb-d2acbb5b08f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:25.005596 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb6acf-9eb6-4d1b-8dbb-d2acbb5b08f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:25.006336 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb6acf-9eb6-4d1b-8dbb-d2acbb5b08f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:21:25.006642 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb6acf-9eb6-4d1b-8dbb-d2acbb5b08f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:25.008261 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb6acf-9eb6-4d1b-8dbb-d2acbb5b08f5 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:25.019932 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:25.029846 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1410/2832] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:21:25.030965 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07829e4e-deb5-4ed5-ba71-1e581237f007 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:25.041122 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07829e4e-deb5-4ed5-ba71-1e581237f007 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:25.045983 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d1ccd-1b85-4710-8632-9a715400f327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:25.046651 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d1ccd-1b85-4710-8632-9a715400f327 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:25.046974 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d1ccd-1b85-4710-8632-9a715400f327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:25.047257 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d1ccd-1b85-4710-8632-9a715400f327 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:25.047966 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d1ccd-1b85-4710-8632-9a715400f327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:25.049860 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d1ccd-1b85-4710-8632-9a715400f327 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:25.081341 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1423/2833] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:21:25 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:21:25.083338 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb6acf-9eb6-4d1b-8dbb-d2acbb5b08f5 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:25.093766 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb6acf-9eb6-4d1b-8dbb-d2acbb5b08f5 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:25.148398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5037a4-edb4-4c70-95c5-3ad669160794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:25.149083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5037a4-edb4-4c70-95c5-3ad669160794 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:25.149349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5037a4-edb4-4c70-95c5-3ad669160794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:25.149627 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5037a4-edb4-4c70-95c5-3ad669160794 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:25.150305 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5037a4-edb4-4c70-95c5-3ad669160794 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:25.152263 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5037a4-edb4-4c70-95c5-3ad669160794 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:25.172703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1411/2834] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:21:25 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:21:25.176819 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d1ccd-1b85-4710-8632-9a715400f327 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:25.187300 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d1ccd-1b85-4710-8632-9a715400f327 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:25.191821 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea40f5c-8978-459a-979d-e234e515efb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:25.192479 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea40f5c-8978-459a-979d-e234e515efb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:25.192761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea40f5c-8978-459a-979d-e234e515efb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:25.193036 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea40f5c-8978-459a-979d-e234e515efb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:25.193761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea40f5c-8978-459a-979d-e234e515efb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:25.195378 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea40f5c-8978-459a-979d-e234e515efb8 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:25.206602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1424/2835] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:21:25.210530 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5037a4-edb4-4c70-95c5-3ad669160794 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:25.220993 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5037a4-edb4-4c70-95c5-3ad669160794 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:25.225513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213a9a0-9ccd-4536-b167-52331572ed1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:25.226173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213a9a0-9ccd-4536-b167-52331572ed1f admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:25.226453 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213a9a0-9ccd-4536-b167-52331572ed1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:25.226828 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213a9a0-9ccd-4536-b167-52331572ed1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:25.227584 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213a9a0-9ccd-4536-b167-52331572ed1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:25.229157 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213a9a0-9ccd-4536-b167-52331572ed1f admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:25.239172 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1412/2836] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:21:25.243088 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea40f5c-8978-459a-979d-e234e515efb8 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:25.253644 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea40f5c-8978-459a-979d-e234e515efb8 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:25.258172 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9bd97a-5c8b-4ce9-873e-96273e63ed60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:25.259196 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bd97a-5c8b-4ce9-873e-96273e63ed60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:25.259487 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bd97a-5c8b-4ce9-873e-96273e63ed60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:25.259770 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bd97a-5c8b-4ce9-873e-96273e63ed60 admin admin] PATH_INFO: `/v3/projects/b68b4f0ba7c74ea6b7ff30d498472485/users/af5551875c7e4f7ab8fe9fe9a23cbdca/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:25.274761 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9bd97a-5c8b-4ce9-873e-96273e63ed60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68b4f0ba7c74ea6b7ff30d498472485, user_id=af5551875c7e4f7ab8fe9fe9a23cbdca, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:25.277400 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9bd97a-5c8b-4ce9-873e-96273e63ed60 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:25.291192 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1425/2837] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:21:25 2022] PUT /identity/v3/projects/b68b4f0ba7c74ea6b7ff30d498472485/users/af5551875c7e4f7ab8fe9fe9a23cbdca/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:21:25.302144 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebd8f5-c998-4914-9c33-89218bcf596d None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:25.302655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebd8f5-c998-4914-9c33-89218bcf596d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:25.302655 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebd8f5-c998-4914-9c33-89218bcf596d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:25.333193 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-2bebd8f5-c998-4914-9c33-89218bcf596d None None] MFA Rules not processed for user `af5551875c7e4f7ab8fe9fe9a23cbdca`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:25.334425 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-2bebd8f5-c998-4914-9c33-89218bcf596d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:25.400458 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1413/2838] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:21:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:21:25.411971 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9bd97a-5c8b-4ce9-873e-96273e63ed60 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:25.413616 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-3f9bd97a-5c8b-4ce9-873e-96273e63ed60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:25.428055 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:25.472875 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9bd97a-5c8b-4ce9-873e-96273e63ed60 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:25.476516 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3105b9-b53b-46a0-b75e-8b1ae93a1a55 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb93eb1d25c4410591125b0ab7a73149', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:25.477164 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3105b9-b53b-46a0-b75e-8b1ae93a1a55 service glance] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:25.477449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3105b9-b53b-46a0-b75e-8b1ae93a1a55 service glance] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:25.477728 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3105b9-b53b-46a0-b75e-8b1ae93a1a55 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:25.486203 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3105b9-b53b-46a0-b75e-8b1ae93a1a55 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:25.487678 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3105b9-b53b-46a0-b75e-8b1ae93a1a55 service glance] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:25.496328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1426/2839] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:21:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:21:26.446385 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebd8f5-c998-4914-9c33-89218bcf596d None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:26.447820 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-2bebd8f5-c998-4914-9c33-89218bcf596d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:26.524361 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:26.527316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebd8f5-c998-4914-9c33-89218bcf596d None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:26.532767 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b09a9-29f5-475b-a3f2-d5a206dc4866 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:26.533616 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b09a9-29f5-475b-a3f2-d5a206dc4866 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:26.534175 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b09a9-29f5-475b-a3f2-d5a206dc4866 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:26.534457 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b09a9-29f5-475b-a3f2-d5a206dc4866 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:26.536702 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5c9b09a9-29f5-475b-a3f2-d5a206dc4866 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:26.602757 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b09a9-29f5-475b-a3f2-d5a206dc4866 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:26.604204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b09a9-29f5-475b-a3f2-d5a206dc4866 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:26.614539 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1414/2840] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:21:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:21:28.247984 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3105b9-b53b-46a0-b75e-8b1ae93a1a55 service glance] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:28.250349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-6c3105b9-b53b-46a0-b75e-8b1ae93a1a55 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:28.417039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3105b9-b53b-46a0-b75e-8b1ae93a1a55 service glance] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:28.425174 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54862cfc-206e-4fcb-8c5f-07394f0f94ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:28.425865 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54862cfc-206e-4fcb-8c5f-07394f0f94ed service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:28.426221 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54862cfc-206e-4fcb-8c5f-07394f0f94ed service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:28.426492 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54862cfc-206e-4fcb-8c5f-07394f0f94ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:28.429723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-54862cfc-206e-4fcb-8c5f-07394f0f94ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:28.444202 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54862cfc-206e-4fcb-8c5f-07394f0f94ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:28.445831 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54862cfc-206e-4fcb-8c5f-07394f0f94ed service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:28.458533 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1427/2841] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:21:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:21:36.610156 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=28 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:38.453437 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=63 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:21:50.223470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b09a9-29f5-475b-a3f2-d5a206dc4866 service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:50.285508 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b09a9-29f5-475b-a3f2-d5a206dc4866 service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:50.290080 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e60637-ff19-4e10-b2ef-4948bf813b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:50.290999 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e60637-ff19-4e10-b2ef-4948bf813b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:50.291286 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e60637-ff19-4e10-b2ef-4948bf813b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:50.291566 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e60637-ff19-4e10-b2ef-4948bf813b08 admin admin] PATH_INFO: `/v3/users/af5551875c7e4f7ab8fe9fe9a23cbdca` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:50.292751 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e60637-ff19-4e10-b2ef-4948bf813b08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af5551875c7e4f7ab8fe9fe9a23cbdca)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:50.294911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e60637-ff19-4e10-b2ef-4948bf813b08 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:50.342006 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-68e60637-ff19-4e10-b2ef-4948bf813b08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af5551875c7e4f7ab8fe9fe9a23cbdca'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:50.354018 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-68e60637-ff19-4e10-b2ef-4948bf813b08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af5551875c7e4f7ab8fe9fe9a23cbdca'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:50.361927 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-68e60637-ff19-4e10-b2ef-4948bf813b08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af5551875c7e4f7ab8fe9fe9a23cbdca'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:50.362624 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-68e60637-ff19-4e10-b2ef-4948bf813b08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af5551875c7e4f7ab8fe9fe9a23cbdca'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:50.370161 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-68e60637-ff19-4e10-b2ef-4948bf813b08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af5551875c7e4f7ab8fe9fe9a23cbdca'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:50.376696 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1415/2842] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:21:50 2022] DELETE /identity/v3/users/af5551875c7e4f7ab8fe9fe9a23cbdca => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:21:50.381236 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5cfa9-01a6-4ffe-bfd5-f81c0e1fb408 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:50.381727 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5cfa9-01a6-4ffe-bfd5-f81c0e1fb408 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:50.381835 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5cfa9-01a6-4ffe-bfd5-f81c0e1fb408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:50.399396 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-8eb5cfa9-01a6-4ffe-bfd5-f81c0e1fb408 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:21:50.400592 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8eb5cfa9-01a6-4ffe-bfd5-f81c0e1fb408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:50.459265 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1428/2843] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:21:50.469158 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e60637-ff19-4e10-b2ef-4948bf813b08 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:50.470609 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-68e60637-ff19-4e10-b2ef-4948bf813b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:50.528410 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e60637-ff19-4e10-b2ef-4948bf813b08 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:50.532076 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599564c3-a151-4abc-b396-531ea8d7917e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:50.532715 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599564c3-a151-4abc-b396-531ea8d7917e service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:50.532994 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599564c3-a151-4abc-b396-531ea8d7917e service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:50.533266 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599564c3-a151-4abc-b396-531ea8d7917e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:50.542532 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599564c3-a151-4abc-b396-531ea8d7917e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:50.544513 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599564c3-a151-4abc-b396-531ea8d7917e service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:50.554135 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1416/2844] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:21:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:21:50.820065 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5cfa9-01a6-4ffe-bfd5-f81c0e1fb408 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:21:50.821718 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-8eb5cfa9-01a6-4ffe-bfd5-f81c0e1fb408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:21:50.835012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5cfa9-01a6-4ffe-bfd5-f81c0e1fb408 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:21:50.840176 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c3a06-ab5f-4500-8eda-722ecfc74ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:21:50.842127 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c3a06-ab5f-4500-8eda-722ecfc74ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:21:50.842414 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c3a06-ab5f-4500-8eda-722ecfc74ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:21:50.842722 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c3a06-ab5f-4500-8eda-722ecfc74ef4 admin admin] PATH_INFO: `/v3/projects/b68b4f0ba7c74ea6b7ff30d498472485` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:21:50.844214 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c3a06-ab5f-4500-8eda-722ecfc74ef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b68b4f0ba7c74ea6b7ff30d498472485)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:21:50.845843 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c3a06-ab5f-4500-8eda-722ecfc74ef4 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:21:50.894527 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2b8c3a06-ab5f-4500-8eda-722ecfc74ef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b68b4f0ba7c74ea6b7ff30d498472485', 'request_id': 'req-2b8c3a06-ab5f-4500-8eda-722ecfc74ef4'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:50.900377 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2b8c3a06-ab5f-4500-8eda-722ecfc74ef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b68b4f0ba7c74ea6b7ff30d498472485', 'request_id': 'req-2b8c3a06-ab5f-4500-8eda-722ecfc74ef4'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:50.907097 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2b8c3a06-ab5f-4500-8eda-722ecfc74ef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b68b4f0ba7c74ea6b7ff30d498472485', 'request_id': 'req-2b8c3a06-ab5f-4500-8eda-722ecfc74ef4'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:50.920398 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2b8c3a06-ab5f-4500-8eda-722ecfc74ef4 admin admin] The token cache is being invalidate because project b68b4f0ba7c74ea6b7ff30d498472485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:21:50.920694 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2b8c3a06-ab5f-4500-8eda-722ecfc74ef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:50.921326 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-2b8c3a06-ab5f-4500-8eda-722ecfc74ef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:21:50.922918 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1429/2845] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:21:50 2022] DELETE /identity/v3/projects/b68b4f0ba7c74ea6b7ff30d498472485 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:22:00.549843 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:00.916197 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:25.244848 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f2b76-c9bb-4cd5-9cdf-686b040e401e None None] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.245449 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f2b76-c9bb-4cd5-9cdf-686b040e401e None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.245449 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f2b76-c9bb-4cd5-9cdf-686b040e401e None None] PATH_INFO: `/` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.247662 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1417/2846] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jul 20 09:22:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:25.253936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c79cc-46e9-4713-90ab-b2a0fb7f353d None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.254422 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c79cc-46e9-4713-90ab-b2a0fb7f353d None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.254531 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c79cc-46e9-4713-90ab-b2a0fb7f353d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.273342 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-834c79cc-46e9-4713-90ab-b2a0fb7f353d None None] MFA Rules not processed for user `5e5ad4e122ae471cad4cf1b0e3111dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:25.274558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-834c79cc-46e9-4713-90ab-b2a0fb7f353d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:25.332045 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1430/2847] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jul 20 09:22:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:22:25.344339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f2b76-c9bb-4cd5-9cdf-686b040e401e None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:25.346136 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a76f2b76-c9bb-4cd5-9cdf-686b040e401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:25.364876 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f2b76-c9bb-4cd5-9cdf-686b040e401e None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:25.368510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e0315-1d23-44f0-a4dd-cb4296972c49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:25.369160 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e0315-1d23-44f0-a4dd-cb4296972c49 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.369534 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e0315-1d23-44f0-a4dd-cb4296972c49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.369805 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e0315-1d23-44f0-a4dd-cb4296972c49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.377312 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e0315-1d23-44f0-a4dd-cb4296972c49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:25.378852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e0315-1d23-44f0-a4dd-cb4296972c49 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:25.387237 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1418/2848] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:22:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:22:25.472432 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fba08b-8742-4fc4-bd97-a3609c527d45 None None] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.472723 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fba08b-8742-4fc4-bd97-a3609c527d45 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.473000 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fba08b-8742-4fc4-bd97-a3609c527d45 None None] PATH_INFO: `/` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.474424 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1431/2849] 10.222.0.44 () {54 vars in 921 bytes} [Wed Jul 20 09:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 09:22:25.479770 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b332fb-fbd4-49d6-b003-72816b584ae8 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.480087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b332fb-fbd4-49d6-b003-72816b584ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.480398 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b332fb-fbd4-49d6-b003-72816b584ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.498332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-10b332fb-fbd4-49d6-b003-72816b584ae8 None None] MFA Rules not processed for user `be08c43f0d0c4a8194596e54500b44ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:25.499546 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-10b332fb-fbd4-49d6-b003-72816b584ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:25.568862 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1419/2850] 10.222.0.44 () {58 vars in 1018 bytes} [Wed Jul 20 09:22:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:22:25.579976 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fba08b-8742-4fc4-bd97-a3609c527d45 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:25.591598 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fba08b-8742-4fc4-bd97-a3609c527d45 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:25.595163 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ef3b5-3240-49a7-ba97-320e32364a88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:25.595738 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ef3b5-3240-49a7-ba97-320e32364a88 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:25.596018 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ef3b5-3240-49a7-ba97-320e32364a88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:25.596295 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ef3b5-3240-49a7-ba97-320e32364a88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:25.606698 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ef3b5-3240-49a7-ba97-320e32364a88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:25.608123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ef3b5-3240-49a7-ba97-320e32364a88 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:25.618692 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1432/2851] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:22:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:22:35.559099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=37 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:35.614074 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=29 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:22:47.526735 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b332fb-fbd4-49d6-b003-72816b584ae8 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:47.528237 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-10b332fb-fbd4-49d6-b003-72816b584ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:47.539629 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b332fb-fbd4-49d6-b003-72816b584ae8 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:47.543191 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ef887-a9f3-4ce6-ab34-beb56d38715d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:47.544003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ef887-a9f3-4ce6-ab34-beb56d38715d service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:47.544291 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ef887-a9f3-4ce6-ab34-beb56d38715d service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:47.544563 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ef887-a9f3-4ce6-ab34-beb56d38715d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:47.546308 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-406ef887-a9f3-4ce6-ab34-beb56d38715d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:47.560431 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ef887-a9f3-4ce6-ab34-beb56d38715d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:47.561852 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ef887-a9f3-4ce6-ab34-beb56d38715d service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:47.572755 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1420/2852] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:22:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:22:53.100462 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ef3b5-3240-49a7-ba97-320e32364a88 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:53.110002 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ef3b5-3240-49a7-ba97-320e32364a88 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:53.113600 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bf7e7-c279-43c3-9b1e-7aba76aed66e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:53.114240 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bf7e7-c279-43c3-9b1e-7aba76aed66e service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:53.114522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bf7e7-c279-43c3-9b1e-7aba76aed66e service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:53.114836 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bf7e7-c279-43c3-9b1e-7aba76aed66e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:53.116582 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-799bf7e7-c279-43c3-9b1e-7aba76aed66e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:53.130630 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bf7e7-c279-43c3-9b1e-7aba76aed66e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:53.132063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bf7e7-c279-43c3-9b1e-7aba76aed66e service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:53.143188 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1433/2853] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:22:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:22:55.462383 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3ad44-dbe3-4ced-9998-4749f6e409e6 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:55.462889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3ad44-dbe3-4ced-9998-4749f6e409e6 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:55.463002 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3ad44-dbe3-4ced-9998-4749f6e409e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:55.480330 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-5ce3ad44-dbe3-4ced-9998-4749f6e409e6 None None] MFA Rules not processed for user `6b8db4a7ee284b1aa96d4517dd381e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:22:55.481508 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5ce3ad44-dbe3-4ced-9998-4749f6e409e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:55.535331 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1421/2854] 10.222.0.44 () {58 vars in 967 bytes} [Wed Jul 20 09:22:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:22:55.582224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bf7e7-c279-43c3-9b1e-7aba76aed66e service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:22:55.583670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-799bf7e7-c279-43c3-9b1e-7aba76aed66e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:22:55.651915 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bf7e7-c279-43c3-9b1e-7aba76aed66e service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:22:55.657457 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a1c3a-b332-49b8-971f-5d7a8e776daa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03b4375e6ead42fb8d77a7b30617cb95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:22:55.658103 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a1c3a-b332-49b8-971f-5d7a8e776daa service swift] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:22:55.658389 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a1c3a-b332-49b8-971f-5d7a8e776daa service swift] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:22:55.658670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a1c3a-b332-49b8-971f-5d7a8e776daa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:22:55.667792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a1c3a-b332-49b8-971f-5d7a8e776daa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:22:55.670077 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a1c3a-b332-49b8-971f-5d7a8e776daa service swift] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:22:55.678071 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1434/2855] 10.222.0.44 () {60 vars in 1384 bytes} [Wed Jul 20 09:22:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 09:23:02.403456 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3ad44-dbe3-4ced-9998-4749f6e409e6 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:02.404983 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5ce3ad44-dbe3-4ced-9998-4749f6e409e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:02.478179 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3ad44-dbe3-4ced-9998-4749f6e409e6 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:02.482956 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472c9cb-5d6f-4a17-9c76-c9c501787aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:02.484090 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472c9cb-5d6f-4a17-9c76-c9c501787aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:02.484375 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472c9cb-5d6f-4a17-9c76-c9c501787aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:02.484788 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472c9cb-5d6f-4a17-9c76-c9c501787aaf admin admin] PATH_INFO: `/v3/users/5a2f15a789f1482ba7a4ecbd260913ea` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:02.485983 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472c9cb-5d6f-4a17-9c76-c9c501787aaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a2f15a789f1482ba7a4ecbd260913ea)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:02.487943 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472c9cb-5d6f-4a17-9c76-c9c501787aaf admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:02.533934 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8472c9cb-5d6f-4a17-9c76-c9c501787aaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a2f15a789f1482ba7a4ecbd260913ea'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:02.546083 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8472c9cb-5d6f-4a17-9c76-c9c501787aaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a2f15a789f1482ba7a4ecbd260913ea'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:02.552972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8472c9cb-5d6f-4a17-9c76-c9c501787aaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a2f15a789f1482ba7a4ecbd260913ea'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:02.553693 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8472c9cb-5d6f-4a17-9c76-c9c501787aaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a2f15a789f1482ba7a4ecbd260913ea'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:02.561930 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-8472c9cb-5d6f-4a17-9c76-c9c501787aaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a2f15a789f1482ba7a4ecbd260913ea'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:02.568252 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1422/2856] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:23:02 2022] DELETE /identity/v3/users/5a2f15a789f1482ba7a4ecbd260913ea => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:23:02.735569 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a1c3a-b332-49b8-971f-5d7a8e776daa service swift] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:02.746412 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a1c3a-b332-49b8-971f-5d7a8e776daa service swift] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:02.806031 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8920c4f-8389-409d-a627-471fe2087e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:02.806835 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8920c4f-8389-409d-a627-471fe2087e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:02.807145 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8920c4f-8389-409d-a627-471fe2087e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:02.807423 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8920c4f-8389-409d-a627-471fe2087e6c admin admin] PATH_INFO: `/v3/projects/96007c68c7f349a19f6bc8c16f024cc6` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:02.808586 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8920c4f-8389-409d-a627-471fe2087e6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=96007c68c7f349a19f6bc8c16f024cc6)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:02.811306 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8920c4f-8389-409d-a627-471fe2087e6c admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:02.855793 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d8920c4f-8389-409d-a627-471fe2087e6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96007c68c7f349a19f6bc8c16f024cc6', 'request_id': 'req-d8920c4f-8389-409d-a627-471fe2087e6c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:02.861954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d8920c4f-8389-409d-a627-471fe2087e6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96007c68c7f349a19f6bc8c16f024cc6', 'request_id': 'req-d8920c4f-8389-409d-a627-471fe2087e6c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:02.868185 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d8920c4f-8389-409d-a627-471fe2087e6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96007c68c7f349a19f6bc8c16f024cc6', 'request_id': 'req-d8920c4f-8389-409d-a627-471fe2087e6c'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:02.882057 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d8920c4f-8389-409d-a627-471fe2087e6c admin admin] The token cache is being invalidate because project 96007c68c7f349a19f6bc8c16f024cc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:23:02.882349 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d8920c4f-8389-409d-a627-471fe2087e6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:02.883019 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-d8920c4f-8389-409d-a627-471fe2087e6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:02.884588 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1435/2857] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:23:02 2022] DELETE /identity/v3/projects/96007c68c7f349a19f6bc8c16f024cc6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:23:02.888821 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472c9cb-5d6f-4a17-9c76-c9c501787aaf admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:02.890226 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-8472c9cb-5d6f-4a17-9c76-c9c501787aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:02.903959 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472c9cb-5d6f-4a17-9c76-c9c501787aaf admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:02.911803 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:02.962121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2b742-5bfa-4c2c-950d-02b124fcacff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:02.963149 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2b742-5bfa-4c2c-950d-02b124fcacff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:02.963435 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2b742-5bfa-4c2c-950d-02b124fcacff admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:02.963711 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2b742-5bfa-4c2c-950d-02b124fcacff admin admin] PATH_INFO: `/v3/users/50aeb5eaa5b745aeb8e937a27584b606` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:02.964904 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e2b742-5bfa-4c2c-950d-02b124fcacff admin admin] RBAC: Authorizing `identity:delete_user(user_id=50aeb5eaa5b745aeb8e937a27584b606)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:02.966911 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e2b742-5bfa-4c2c-950d-02b124fcacff admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.011720 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-60e2b742-5bfa-4c2c-950d-02b124fcacff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50aeb5eaa5b745aeb8e937a27584b606'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:03.023787 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-60e2b742-5bfa-4c2c-950d-02b124fcacff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50aeb5eaa5b745aeb8e937a27584b606'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:03.031104 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-60e2b742-5bfa-4c2c-950d-02b124fcacff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50aeb5eaa5b745aeb8e937a27584b606'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:03.031780 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-60e2b742-5bfa-4c2c-950d-02b124fcacff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50aeb5eaa5b745aeb8e937a27584b606'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:03.038652 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-60e2b742-5bfa-4c2c-950d-02b124fcacff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50aeb5eaa5b745aeb8e937a27584b606'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:03.045346 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1423/2858] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:23:02 2022] DELETE /identity/v3/users/50aeb5eaa5b745aeb8e937a27584b606 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:23:03.218959 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8920c4f-8389-409d-a627-471fe2087e6c admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.220497 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d8920c4f-8389-409d-a627-471fe2087e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:03.234569 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8920c4f-8389-409d-a627-471fe2087e6c admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.238654 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:03.296620 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa46f1-251f-4912-aab5-329aac753758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.297550 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa46f1-251f-4912-aab5-329aac753758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.297953 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa46f1-251f-4912-aab5-329aac753758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.298279 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa46f1-251f-4912-aab5-329aac753758 admin admin] PATH_INFO: `/v3/projects/263a1059e7af42ed9faaa331a3b4c76a` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.299572 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa46f1-251f-4912-aab5-329aac753758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=263a1059e7af42ed9faaa331a3b4c76a)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.301523 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa46f1-251f-4912-aab5-329aac753758 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.347305 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-93fa46f1-251f-4912-aab5-329aac753758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '263a1059e7af42ed9faaa331a3b4c76a', 'request_id': 'req-93fa46f1-251f-4912-aab5-329aac753758'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:03.352674 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-93fa46f1-251f-4912-aab5-329aac753758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '263a1059e7af42ed9faaa331a3b4c76a', 'request_id': 'req-93fa46f1-251f-4912-aab5-329aac753758'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:03.359634 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-93fa46f1-251f-4912-aab5-329aac753758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '263a1059e7af42ed9faaa331a3b4c76a', 'request_id': 'req-93fa46f1-251f-4912-aab5-329aac753758'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:03.372102 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-93fa46f1-251f-4912-aab5-329aac753758 admin admin] The token cache is being invalidate because project 263a1059e7af42ed9faaa331a3b4c76a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:23:03.372397 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-93fa46f1-251f-4912-aab5-329aac753758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:03.373210 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-93fa46f1-251f-4912-aab5-329aac753758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:23:03.374795 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1436/2859] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:23:03 2022] DELETE /identity/v3/projects/263a1059e7af42ed9faaa331a3b4c76a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:23:03.379856 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf67cff-9df7-48f2-9f73-562571444757 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.380146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf67cff-9df7-48f2-9f73-562571444757 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.380427 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf67cff-9df7-48f2-9f73-562571444757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.398708 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-fbf67cff-9df7-48f2-9f73-562571444757 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:03.399899 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-fbf67cff-9df7-48f2-9f73-562571444757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:03.457607 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1424/2860] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:23:03.465253 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477b256-faa8-4978-b8a5-f89e0452b6b2 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.465606 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477b256-faa8-4978-b8a5-f89e0452b6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.466594 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477b256-faa8-4978-b8a5-f89e0452b6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.484568 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-d477b256-faa8-4978-b8a5-f89e0452b6b2 None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:03.485767 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-d477b256-faa8-4978-b8a5-f89e0452b6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:03.493559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1437/2861] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:23:03.497608 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf67cff-9df7-48f2-9f73-562571444757 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.507168 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf67cff-9df7-48f2-9f73-562571444757 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.511632 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6bbf7-348c-4c65-840f-3ca4122aadda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.512272 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6bbf7-348c-4c65-840f-3ca4122aadda admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.512553 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6bbf7-348c-4c65-840f-3ca4122aadda admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.513360 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6bbf7-348c-4c65-840f-3ca4122aadda admin admin] PATH_INFO: `/v3/domains` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.514072 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6bbf7-348c-4c65-840f-3ca4122aadda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80334) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 09:23:03.514375 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6bbf7-348c-4c65-840f-3ca4122aadda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.516008 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6bbf7-348c-4c65-840f-3ca4122aadda admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.527253 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1425/2862] 10.222.0.44 () {58 vars in 1152 bytes} [Wed Jul 20 09:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 09:23:03.531968 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477b256-faa8-4978-b8a5-f89e0452b6b2 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.541224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477b256-faa8-4978-b8a5-f89e0452b6b2 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.545960 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418e2ea-2f3d-42fe-8999-6f4184e0093f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.546614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418e2ea-2f3d-42fe-8999-6f4184e0093f admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.546936 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418e2ea-2f3d-42fe-8999-6f4184e0093f admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.547212 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418e2ea-2f3d-42fe-8999-6f4184e0093f admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.547926 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418e2ea-2f3d-42fe-8999-6f4184e0093f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.549798 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418e2ea-2f3d-42fe-8999-6f4184e0093f admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.585597 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1438/2863] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:23:03 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:03.589738 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6bbf7-348c-4c65-840f-3ca4122aadda admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.600729 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6bbf7-348c-4c65-840f-3ca4122aadda admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.613146 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:03.657858 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0242c1-6e9c-4356-8585-68aa13d255f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.658525 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0242c1-6e9c-4356-8585-68aa13d255f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.658841 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0242c1-6e9c-4356-8585-68aa13d255f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.659121 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0242c1-6e9c-4356-8585-68aa13d255f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.659807 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0242c1-6e9c-4356-8585-68aa13d255f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.661713 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0242c1-6e9c-4356-8585-68aa13d255f3 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.681667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1426/2864] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:23:03 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:03.685918 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418e2ea-2f3d-42fe-8999-6f4184e0093f admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.696068 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418e2ea-2f3d-42fe-8999-6f4184e0093f admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.701449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229c61a-f1c6-46ac-b9ef-26fce3aef794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.702097 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229c61a-f1c6-46ac-b9ef-26fce3aef794 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.702380 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229c61a-f1c6-46ac-b9ef-26fce3aef794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.702699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229c61a-f1c6-46ac-b9ef-26fce3aef794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.703431 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e229c61a-f1c6-46ac-b9ef-26fce3aef794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.704999 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e229c61a-f1c6-46ac-b9ef-26fce3aef794 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.714737 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1439/2865] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:23:03.719162 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0242c1-6e9c-4356-8585-68aa13d255f3 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.729309 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0242c1-6e9c-4356-8585-68aa13d255f3 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.734111 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b559af-7aba-4b9b-9fc3-6c82cddd9459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.734801 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b559af-7aba-4b9b-9fc3-6c82cddd9459 admin admin] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.735087 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b559af-7aba-4b9b-9fc3-6c82cddd9459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.735365 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b559af-7aba-4b9b-9fc3-6c82cddd9459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.736483 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b559af-7aba-4b9b-9fc3-6c82cddd9459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.738058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b559af-7aba-4b9b-9fc3-6c82cddd9459 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.747872 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1427/2866] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:23:03.751844 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229c61a-f1c6-46ac-b9ef-26fce3aef794 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.761617 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229c61a-f1c6-46ac-b9ef-26fce3aef794 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.766850 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cad1d-5e41-47c7-abde-6ffbfdbba13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.767753 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cad1d-5e41-47c7-abde-6ffbfdbba13a admin admin] REQUEST_METHOD: `PUT` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.768115 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cad1d-5e41-47c7-abde-6ffbfdbba13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.768302 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cad1d-5e41-47c7-abde-6ffbfdbba13a admin admin] PATH_INFO: `/v3/projects/a93c746bc7844fbfad63dc528ae346f7/users/ae24e34d799e4db292be714fd49579c7/roles/0e2011aac3e1430cb96bdbf810e89eb7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.784014 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cad1d-5e41-47c7-abde-6ffbfdbba13a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a93c746bc7844fbfad63dc528ae346f7, user_id=ae24e34d799e4db292be714fd49579c7, role_id=0e2011aac3e1430cb96bdbf810e89eb7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.786698 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cad1d-5e41-47c7-abde-6ffbfdbba13a admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.799364 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1440/2867] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:23:03 2022] PUT /identity/v3/projects/a93c746bc7844fbfad63dc528ae346f7/users/ae24e34d799e4db292be714fd49579c7/roles/0e2011aac3e1430cb96bdbf810e89eb7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:23:03.804463 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ba4e7-7741-433d-8a64-15bb7fa0782d None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.804756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ba4e7-7741-433d-8a64-15bb7fa0782d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.805059 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ba4e7-7741-433d-8a64-15bb7fa0782d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.822857 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-4c2ba4e7-7741-433d-8a64-15bb7fa0782d None None] MFA Rules not processed for user `d01511dd98c7416fb8387a8557165556`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:03.824036 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-4c2ba4e7-7741-433d-8a64-15bb7fa0782d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:03.883666 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1428/2868] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:23:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:23:03.894075 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cad1d-5e41-47c7-abde-6ffbfdbba13a admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:03.895833 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-ba9cad1d-5e41-47c7-abde-6ffbfdbba13a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:03.925416 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:03.958193 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cad1d-5e41-47c7-abde-6ffbfdbba13a admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:03.961952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f24460-c081-4024-aa62-1f33ddfd1cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:03.962605 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f24460-c081-4024-aa62-1f33ddfd1cac service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:03.962937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f24460-c081-4024-aa62-1f33ddfd1cac service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:03.963219 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f24460-c081-4024-aa62-1f33ddfd1cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:03.973109 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f24460-c081-4024-aa62-1f33ddfd1cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:03.974535 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f24460-c081-4024-aa62-1f33ddfd1cac service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:03.984393 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1441/2869] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:23:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:23:07.717810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f1d7a-5f61-429c-bc4e-2932231d702d None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:07.718790 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f1d7a-5f61-429c-bc4e-2932231d702d None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:07.719284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f1d7a-5f61-429c-bc4e-2932231d702d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:07.747809 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-370f1d7a-5f61-429c-bc4e-2932231d702d None None] MFA Rules not processed for user `ae24e34d799e4db292be714fd49579c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:07.749317 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-370f1d7a-5f61-429c-bc4e-2932231d702d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:07.817697 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1429/2870] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:23:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:23:07.822389 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f24460-c081-4024-aa62-1f33ddfd1cac service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:07.833834 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f24460-c081-4024-aa62-1f33ddfd1cac service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:07.838643 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa086a66-7608-44a2-91b3-866039f690a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:07.839415 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa086a66-7608-44a2-91b3-866039f690a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:07.839769 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa086a66-7608-44a2-91b3-866039f690a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:07.840117 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa086a66-7608-44a2-91b3-866039f690a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:07.841095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa086a66-7608-44a2-91b3-866039f690a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:07.843114 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa086a66-7608-44a2-91b3-866039f690a0 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:07.877403 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1442/2871] 10.222.0.44 () {60 vars in 1152 bytes} [Wed Jul 20 09:23:07 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 09:23:07.879431 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f1d7a-5f61-429c-bc4e-2932231d702d None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:07.901737 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f1d7a-5f61-429c-bc4e-2932231d702d None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:07.943282 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:07.983284 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff46d12-17de-45df-89f5-16a32948ad97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:07.983981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff46d12-17de-45df-89f5-16a32948ad97 admin admin] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:07.984269 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff46d12-17de-45df-89f5-16a32948ad97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:07.984543 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff46d12-17de-45df-89f5-16a32948ad97 admin admin] PATH_INFO: `/v3/users` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:07.985273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff46d12-17de-45df-89f5-16a32948ad97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:07.987279 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff46d12-17de-45df-89f5-16a32948ad97 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:08.014141 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1430/2872] 10.222.0.44 () {60 vars in 1143 bytes} [Wed Jul 20 09:23:07 2022] POST /identity/v3/users => generated 295 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 09:23:08.018180 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa086a66-7608-44a2-91b3-866039f690a0 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:08.030695 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa086a66-7608-44a2-91b3-866039f690a0 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:08.036000 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e8d44-ea72-4e83-a00d-d3e4c6687192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:08.036762 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e8d44-ea72-4e83-a00d-d3e4c6687192 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:08.037109 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e8d44-ea72-4e83-a00d-d3e4c6687192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:08.037392 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e8d44-ea72-4e83-a00d-d3e4c6687192 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:08.038138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e8d44-ea72-4e83-a00d-d3e4c6687192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:08.041281 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e8d44-ea72-4e83-a00d-d3e4c6687192 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:08.053909 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1443/2873] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 09:23:08.058239 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff46d12-17de-45df-89f5-16a32948ad97 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:08.073173 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff46d12-17de-45df-89f5-16a32948ad97 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:08.079130 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f1fe2-3c8e-4620-8c53-2428711e07ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:08.080037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1fe2-3c8e-4620-8c53-2428711e07ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:08.080322 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1fe2-3c8e-4620-8c53-2428711e07ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:08.080602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1fe2-3c8e-4620-8c53-2428711e07ae admin admin] PATH_INFO: `/v3/projects/239abe6695734d3b977851f9e3721225/users/1fdbb89cb94b465a9ca74a3a4055206f/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:08.098625 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f1fe2-3c8e-4620-8c53-2428711e07ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=239abe6695734d3b977851f9e3721225, user_id=1fdbb89cb94b465a9ca74a3a4055206f, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:08.101339 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f1fe2-3c8e-4620-8c53-2428711e07ae admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:08.118483 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1431/2874] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:23:08 2022] PUT /identity/v3/projects/239abe6695734d3b977851f9e3721225/users/1fdbb89cb94b465a9ca74a3a4055206f/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:23:08.122859 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e8d44-ea72-4e83-a00d-d3e4c6687192 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:08.134947 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e8d44-ea72-4e83-a00d-d3e4c6687192 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:08.198128 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb2512-d1e0-4b4d-bb70-6f04c5da4451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:08.198851 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb2512-d1e0-4b4d-bb70-6f04c5da4451 admin admin] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:08.199138 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb2512-d1e0-4b4d-bb70-6f04c5da4451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:08.199416 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb2512-d1e0-4b4d-bb70-6f04c5da4451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:08.200187 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb2512-d1e0-4b4d-bb70-6f04c5da4451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:08.202037 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb2512-d1e0-4b4d-bb70-6f04c5da4451 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:08.212244 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1444/2875] 10.222.0.44 () {58 vars in 1121 bytes} [Wed Jul 20 09:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 09:23:08.216648 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f1fe2-3c8e-4620-8c53-2428711e07ae admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:08.228233 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f1fe2-3c8e-4620-8c53-2428711e07ae admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:08.234066 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4684f25-ff30-49e8-bfb2-4c9dd68d8093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:08.235126 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4684f25-ff30-49e8-bfb2-4c9dd68d8093 admin admin] REQUEST_METHOD: `PUT` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:08.235472 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4684f25-ff30-49e8-bfb2-4c9dd68d8093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:08.235810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4684f25-ff30-49e8-bfb2-4c9dd68d8093 admin admin] PATH_INFO: `/v3/projects/239abe6695734d3b977851f9e3721225/users/1fdbb89cb94b465a9ca74a3a4055206f/roles/41447ccf04114fd2b491c81d6f43fd8c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:08.237810 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4684f25-ff30-49e8-bfb2-4c9dd68d8093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=239abe6695734d3b977851f9e3721225, user_id=1fdbb89cb94b465a9ca74a3a4055206f, role_id=41447ccf04114fd2b491c81d6f43fd8c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:08.240565 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4684f25-ff30-49e8-bfb2-4c9dd68d8093 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:08.252382 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1432/2876] 10.222.0.44 () {60 vars in 1482 bytes} [Wed Jul 20 09:23:08 2022] PUT /identity/v3/projects/239abe6695734d3b977851f9e3721225/users/1fdbb89cb94b465a9ca74a3a4055206f/roles/41447ccf04114fd2b491c81d6f43fd8c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:23:12.805954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dc7aa-eca8-487f-9c0e-ddeef206fb6b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:12.807000 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dc7aa-eca8-487f-9c0e-ddeef206fb6b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:12.807510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dc7aa-eca8-487f-9c0e-ddeef206fb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:12.843941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-f52dc7aa-eca8-487f-9c0e-ddeef206fb6b None None] MFA Rules not processed for user `1fdbb89cb94b465a9ca74a3a4055206f`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:12.845188 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-f52dc7aa-eca8-487f-9c0e-ddeef206fb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:12.924272 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1445/2877] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:23:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:23:12.941990 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4684f25-ff30-49e8-bfb2-4c9dd68d8093 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:12.944667 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-a4684f25-ff30-49e8-bfb2-4c9dd68d8093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:13.024466 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4684f25-ff30-49e8-bfb2-4c9dd68d8093 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:13.031674 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ab290-8d3d-49b4-b964-cbf2e9b7e4bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:13.032459 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ab290-8d3d-49b4-b964-cbf2e9b7e4bd service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:13.032754 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ab290-8d3d-49b4-b964-cbf2e9b7e4bd service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:13.033038 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ab290-8d3d-49b4-b964-cbf2e9b7e4bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:13.099450 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ab290-8d3d-49b4-b964-cbf2e9b7e4bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:13.101070 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ab290-8d3d-49b4-b964-cbf2e9b7e4bd service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:13.111067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1433/2878] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:23:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:23:22.921052 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:23.106638 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=88,keystone:INSERT=5 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:28.258111 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec5b50-91bd-4940-9939-b8c1d1b1c23b None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:28.258611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec5b50-91bd-4940-9939-b8c1d1b1c23b None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:28.258765 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec5b50-91bd-4940-9939-b8c1d1b1c23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:28.276602 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-66ec5b50-91bd-4940-9939-b8c1d1b1c23b None None] MFA Rules not processed for user `ae24e34d799e4db292be714fd49579c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:23:28.278449 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-66ec5b50-91bd-4940-9939-b8c1d1b1c23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:28.285374 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1446/2879] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:23:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:23:29.037598 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ab290-8d3d-49b4-b964-cbf2e9b7e4bd service nova] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:29.039020 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-373ab290-8d3d-49b4-b964-cbf2e9b7e4bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:29.116413 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ab290-8d3d-49b4-b964-cbf2e9b7e4bd service nova] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:29.121373 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46dc90-5639-42ac-ad0f-4f73e1468031 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '066f69a87f5f48c7b20c1e0fcf5c1107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:29.122015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46dc90-5639-42ac-ad0f-4f73e1468031 service placement] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:29.122449 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46dc90-5639-42ac-ad0f-4f73e1468031 service placement] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:29.122759 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46dc90-5639-42ac-ad0f-4f73e1468031 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:29.124510 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5e46dc90-5639-42ac-ad0f-4f73e1468031 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:29.135723 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46dc90-5639-42ac-ad0f-4f73e1468031 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:29.138185 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46dc90-5639-42ac-ad0f-4f73e1468031 service placement] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:29.148049 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1434/2880] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:23:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:23:30.752614 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec5b50-91bd-4940-9939-b8c1d1b1c23b None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:30.810858 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec5b50-91bd-4940-9939-b8c1d1b1c23b None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:30.814466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721a8d2-223f-43b3-9d01-37ed48e0ac13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:30.815159 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721a8d2-223f-43b3-9d01-37ed48e0ac13 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:30.815450 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721a8d2-223f-43b3-9d01-37ed48e0ac13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:30.815735 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721a8d2-223f-43b3-9d01-37ed48e0ac13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:30.817528 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-2721a8d2-223f-43b3-9d01-37ed48e0ac13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:30.828332 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721a8d2-223f-43b3-9d01-37ed48e0ac13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:30.829775 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2721a8d2-223f-43b3-9d01-37ed48e0ac13 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:30.837870 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1447/2881] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:23:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:23:31.709037 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46dc90-5639-42ac-ad0f-4f73e1468031 service placement] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:31.712355 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-5e46dc90-5639-42ac-ad0f-4f73e1468031 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:31.732692 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46dc90-5639-42ac-ad0f-4f73e1468031 service placement] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:31.738408 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d9f2a-ba81-44b6-b6e1-e1040357a05a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:31.739047 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d9f2a-ba81-44b6-b6e1-e1040357a05a service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:31.739332 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d9f2a-ba81-44b6-b6e1-e1040357a05a service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:31.739610 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d9f2a-ba81-44b6-b6e1-e1040357a05a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:31.741444 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-651d9f2a-ba81-44b6-b6e1-e1040357a05a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:31.755067 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d9f2a-ba81-44b6-b6e1-e1040357a05a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:31.756678 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d9f2a-ba81-44b6-b6e1-e1040357a05a service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:31.769400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1435/2882] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:23:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:23:40.834162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=29 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:41.182505 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721a8d2-223f-43b3-9d01-37ed48e0ac13 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:41.196488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721a8d2-223f-43b3-9d01-37ed48e0ac13 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:41.202328 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98645d4c-e645-4c8a-8729-cf1ebfecf030 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:41.203007 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98645d4c-e645-4c8a-8729-cf1ebfecf030 service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:41.203293 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98645d4c-e645-4c8a-8729-cf1ebfecf030 service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:41.203570 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98645d4c-e645-4c8a-8729-cf1ebfecf030 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:41.205482 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-98645d4c-e645-4c8a-8729-cf1ebfecf030 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:41.218099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98645d4c-e645-4c8a-8729-cf1ebfecf030 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:41.219699 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98645d4c-e645-4c8a-8729-cf1ebfecf030 service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:41.230792 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1448/2883] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:23:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:23:41.764015 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=38 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:43.237474 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d9f2a-ba81-44b6-b6e1-e1040357a05a service nova] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:23:43.238936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-651d9f2a-ba81-44b6-b6e1-e1040357a05a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:23:43.250289 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d9f2a-ba81-44b6-b6e1-e1040357a05a service nova] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:23:43.253831 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b68e8-3622-4790-929a-b4f0035144cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:23:43.254464 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b68e8-3622-4790-929a-b4f0035144cf service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:23:43.254779 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b68e8-3622-4790-929a-b4f0035144cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:23:43.255058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b68e8-3622-4790-929a-b4f0035144cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:23:43.264985 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b68e8-3622-4790-929a-b4f0035144cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:23:43.266401 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b68e8-3622-4790-929a-b4f0035144cf service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:23:43.275747 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1436/2884] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:23:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:23:51.227040 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=6 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:23:53.271617 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=6 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:04.666495 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98645d4c-e645-4c8a-8729-cf1ebfecf030 service nova] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:04.668093 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-98645d4c-e645-4c8a-8729-cf1ebfecf030 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:04.739110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98645d4c-e645-4c8a-8729-cf1ebfecf030 service nova] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:04.743623 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e43c4f-ee69-4127-99a9-020bcab58110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:04.744488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e43c4f-ee69-4127-99a9-020bcab58110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:04.744783 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e43c4f-ee69-4127-99a9-020bcab58110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:04.745039 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e43c4f-ee69-4127-99a9-020bcab58110 admin admin] PATH_INFO: `/v3/users/bcc742eb9e7b42d9bb41f04bb58038db` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:04.746814 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e43c4f-ee69-4127-99a9-020bcab58110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcc742eb9e7b42d9bb41f04bb58038db)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:04.748709 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e43c4f-ee69-4127-99a9-020bcab58110 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:04.792693 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-09e43c4f-ee69-4127-99a9-020bcab58110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcc742eb9e7b42d9bb41f04bb58038db'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:04.804162 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-09e43c4f-ee69-4127-99a9-020bcab58110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcc742eb9e7b42d9bb41f04bb58038db'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:04.811564 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-09e43c4f-ee69-4127-99a9-020bcab58110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcc742eb9e7b42d9bb41f04bb58038db'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:04.812231 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-09e43c4f-ee69-4127-99a9-020bcab58110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcc742eb9e7b42d9bb41f04bb58038db'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:04.819677 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-09e43c4f-ee69-4127-99a9-020bcab58110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcc742eb9e7b42d9bb41f04bb58038db'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:04.826757 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1449/2885] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:24:04 2022] DELETE /identity/v3/users/bcc742eb9e7b42d9bb41f04bb58038db => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:24:04.996703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b68e8-3622-4790-929a-b4f0035144cf service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:05.008574 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b68e8-3622-4790-929a-b4f0035144cf service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:05.067686 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:05.068580 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:05.068868 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:05.069151 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a admin admin] PATH_INFO: `/v3/projects/e6a9958d940546b99e37b85acc95ad7c` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:05.070422 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6a9958d940546b99e37b85acc95ad7c)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:05.072400 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:05.117540 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6a9958d940546b99e37b85acc95ad7c', 'request_id': 'req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:05.123587 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6a9958d940546b99e37b85acc95ad7c', 'request_id': 'req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:05.130964 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6a9958d940546b99e37b85acc95ad7c', 'request_id': 'req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:05.142027 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a admin admin] The token cache is being invalidate because project e6a9958d940546b99e37b85acc95ad7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:24:05.142316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:05.142972 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:05.144490 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1437/2886] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:24:04 2022] DELETE /identity/v3/projects/e6a9958d940546b99e37b85acc95ad7c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:24:05.148438 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e43c4f-ee69-4127-99a9-020bcab58110 admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:05.149842 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-09e43c4f-ee69-4127-99a9-020bcab58110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:05.163058 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e43c4f-ee69-4127-99a9-020bcab58110 admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:05.220200 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b236da-4bf8-47fc-936a-c044a51e4108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:05.220996 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b236da-4bf8-47fc-936a-c044a51e4108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:05.221280 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b236da-4bf8-47fc-936a-c044a51e4108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:05.221558 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b236da-4bf8-47fc-936a-c044a51e4108 admin admin] PATH_INFO: `/v3/users/9386ad7adc354126b5701dda57f2def5` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:05.222785 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b236da-4bf8-47fc-936a-c044a51e4108 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9386ad7adc354126b5701dda57f2def5)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:05.224690 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b236da-4bf8-47fc-936a-c044a51e4108 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:05.248429 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:05.272238 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c1b236da-4bf8-47fc-936a-c044a51e4108 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9386ad7adc354126b5701dda57f2def5'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:05.284682 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c1b236da-4bf8-47fc-936a-c044a51e4108 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9386ad7adc354126b5701dda57f2def5'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:05.291466 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c1b236da-4bf8-47fc-936a-c044a51e4108 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9386ad7adc354126b5701dda57f2def5'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:05.292131 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c1b236da-4bf8-47fc-936a-c044a51e4108 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9386ad7adc354126b5701dda57f2def5'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:05.300038 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-c1b236da-4bf8-47fc-936a-c044a51e4108 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9386ad7adc354126b5701dda57f2def5'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:05.305891 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1450/2887] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:24:05 2022] DELETE /identity/v3/users/9386ad7adc354126b5701dda57f2def5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:24:05.478709 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:24:05.481624 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:24:05.493349 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3b99d-4b05-4f32-a6c4-1f95e2e2a63a admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:24:05.550531 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:24:05.551428 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:24:05.551707 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:24:05.551981 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a admin admin] PATH_INFO: `/v3/projects/ab5e0c8af01e445eb73ffb61ed0fef66` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:24:05.553347 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab5e0c8af01e445eb73ffb61ed0fef66)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:24:05.555088 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:24:05.588637 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:05.602416 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab5e0c8af01e445eb73ffb61ed0fef66', 'request_id': 'req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:05.608571 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab5e0c8af01e445eb73ffb61ed0fef66', 'request_id': 'req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:05.615590 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab5e0c8af01e445eb73ffb61ed0fef66', 'request_id': 'req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:05.626399 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a admin admin] The token cache is being invalidate because project ab5e0c8af01e445eb73ffb61ed0fef66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80334) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:24:05.626703 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:05.627333 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:24:05.628870 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1438/2888] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:24:05 2022] DELETE /identity/v3/projects/ab5e0c8af01e445eb73ffb61ed0fef66 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:24:15.302369 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:24:15.623523 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:45.005543 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbe33b-559d-4807-b585-386cc7048b31 None None] REQUEST_METHOD: `POST` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:45.006072 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbe33b-559d-4807-b585-386cc7048b31 None None] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:45.006175 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbe33b-559d-4807-b585-386cc7048b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:45.024725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.auth.core [None req-7cdbe33b-559d-4807-b585-386cc7048b31 None None] MFA Rules not processed for user `ae24e34d799e4db292be714fd49579c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80333) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:25:45.025941 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7cdbe33b-559d-4807-b585-386cc7048b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:25:45.082243 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1451/2889] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:25:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:25:45.402061 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:25:45.403490 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:25:45.461864 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb7388-42d0-4790-8777-72f7a7e4cb9a admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:25:45.465411 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f61e7b-9091-47e6-a0b5-9eeda4ae5c77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:25:45.466058 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f61e7b-9091-47e6-a0b5-9eeda4ae5c77 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:25:45.466338 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f61e7b-9091-47e6-a0b5-9eeda4ae5c77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:25:45.466613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f61e7b-9091-47e6-a0b5-9eeda4ae5c77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:25:45.468509 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d2f61e7b-9091-47e6-a0b5-9eeda4ae5c77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:25:45.478905 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f61e7b-9091-47e6-a0b5-9eeda4ae5c77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:25:45.480355 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f61e7b-9091-47e6-a0b5-9eeda4ae5c77 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:25:45.489221 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1439/2890] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:25:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:25:55.074486 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=25 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:25:55.484790 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=26 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:26:50.047095 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbe33b-559d-4807-b585-386cc7048b31 None None] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:26:50.048483 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-7cdbe33b-559d-4807-b585-386cc7048b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:26:50.118937 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbe33b-559d-4807-b585-386cc7048b31 None None] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:26:50.123827 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fc2ec-2828-48ff-8693-461075beb10f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '066f69a87f5f48c7b20c1e0fcf5c1107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:26:50.124501 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fc2ec-2828-48ff-8693-461075beb10f service placement] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:26:50.124789 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fc2ec-2828-48ff-8693-461075beb10f service placement] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:26:50.125071 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fc2ec-2828-48ff-8693-461075beb10f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:26:50.126924 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-9a2fc2ec-2828-48ff-8693-461075beb10f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:26:50.138105 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fc2ec-2828-48ff-8693-461075beb10f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:26:50.139725 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fc2ec-2828-48ff-8693-461075beb10f service placement] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:26:50.150599 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1452/2891] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:26:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:27:00.145260 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=32 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:39.581697 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36021c72-391d-48c6-894a-62370b482a48 None None] REQUEST_METHOD: `POST` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:39.582196 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36021c72-391d-48c6-894a-62370b482a48 None None] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:39.582303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36021c72-391d-48c6-894a-62370b482a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:39.601053 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.auth.core [None req-36021c72-391d-48c6-894a-62370b482a48 None None] MFA Rules not processed for user `ae24e34d799e4db292be714fd49579c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80334) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 09:27:39.602274 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-36021c72-391d-48c6-894a-62370b482a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:39.609116 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1440/2892] 10.222.0.44 () {58 vars in 957 bytes} [Wed Jul 20 09:27:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 09:27:39.922160 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fc2ec-2828-48ff-8693-461075beb10f service placement] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:27:39.931549 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fc2ec-2828-48ff-8693-461075beb10f service placement] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:27:39.935053 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a2c7b-84c4-4315-8c48-2633dabda271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:27:39.935784 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a2c7b-84c4-4315-8c48-2633dabda271 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:39.936073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a2c7b-84c4-4315-8c48-2633dabda271 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:39.936354 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a2c7b-84c4-4315-8c48-2633dabda271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:39.938123 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-689a2c7b-84c4-4315-8c48-2633dabda271 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:39.947981 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a2c7b-84c4-4315-8c48-2633dabda271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:27:39.949405 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a2c7b-84c4-4315-8c48-2633dabda271 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:27:39.957560 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1453/2893] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:27:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:27:47.369353 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36021c72-391d-48c6-894a-62370b482a48 None None] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:27:47.379196 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36021c72-391d-48c6-894a-62370b482a48 None None] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:27:47.383057 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4d527-0792-4ad7-bc2f-7e26370c7f41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:27:47.383730 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4d527-0792-4ad7-bc2f-7e26370c7f41 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:47.384028 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4d527-0792-4ad7-bc2f-7e26370c7f41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:47.384303 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4d527-0792-4ad7-bc2f-7e26370c7f41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:47.386095 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-22b4d527-0792-4ad7-bc2f-7e26370c7f41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:47.458204 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4d527-0792-4ad7-bc2f-7e26370c7f41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:27:47.459664 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4d527-0792-4ad7-bc2f-7e26370c7f41 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:27:47.470470 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1441/2894] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:27:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:27:49.954012 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=6 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:27:54.427998 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a2c7b-84c4-4315-8c48-2633dabda271 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:27:54.436705 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a2c7b-84c4-4315-8c48-2633dabda271 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:27:54.440312 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba160f-9f7e-4045-a44d-ac4ceba258c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:27:54.440952 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba160f-9f7e-4045-a44d-ac4ceba258c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:54.441234 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba160f-9f7e-4045-a44d-ac4ceba258c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:54.441510 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba160f-9f7e-4045-a44d-ac4ceba258c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:54.444165 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-1eba160f-9f7e-4045-a44d-ac4ceba258c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:54.458073 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba160f-9f7e-4045-a44d-ac4ceba258c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:27:54.459525 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba160f-9f7e-4045-a44d-ac4ceba258c6 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:27:54.470263 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1454/2895] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:27:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:27:56.543377 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4d527-0792-4ad7-bc2f-7e26370c7f41 service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:27:56.544847 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-22b4d527-0792-4ad7-bc2f-7e26370c7f41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:56.559267 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4d527-0792-4ad7-bc2f-7e26370c7f41 service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:27:56.565094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a86d5-6db2-40f5-9421-1ef64cdcd1a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:27:56.565764 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a86d5-6db2-40f5-9421-1ef64cdcd1a2 service nova] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:27:56.566043 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a86d5-6db2-40f5-9421-1ef64cdcd1a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:27:56.566316 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a86d5-6db2-40f5-9421-1ef64cdcd1a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:27:56.568889 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-e83a86d5-6db2-40f5-9421-1ef64cdcd1a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:27:56.581415 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a86d5-6db2-40f5-9421-1ef64cdcd1a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:27:56.583643 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a86d5-6db2-40f5-9421-1ef64cdcd1a2 service nova] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:27:56.594304 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1442/2896] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:27:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:28:04.465454 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=6 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:06.589756 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=43 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:28:47.774756 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba160f-9f7e-4045-a44d-ac4ceba258c6 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:28:47.776186 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-1eba160f-9f7e-4045-a44d-ac4ceba258c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:47.787274 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba160f-9f7e-4045-a44d-ac4ceba258c6 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:28:47.790948 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4274d-cdd9-433d-b354-b1c889085af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:28:47.791593 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4274d-cdd9-433d-b354-b1c889085af1 service neutron] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:28:47.791875 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4274d-cdd9-433d-b354-b1c889085af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:28:47.792150 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4274d-cdd9-433d-b354-b1c889085af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:28:47.793861 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-e3c4274d-cdd9-433d-b354-b1c889085af1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:28:47.803901 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4274d-cdd9-433d-b354-b1c889085af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:28:47.805246 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4274d-cdd9-433d-b354-b1c889085af1 service neutron] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:28:47.813233 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1455/2897] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:28:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:28:57.809754 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=6 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:23.789273 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a86d5-6db2-40f5-9421-1ef64cdcd1a2 service nova] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:23.798613 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a86d5-6db2-40f5-9421-1ef64cdcd1a2 service nova] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:23.803257 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56237cba-77db-42a5-a946-98f0c9c2cfd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:23.803841 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56237cba-77db-42a5-a946-98f0c9c2cfd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:23.804129 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56237cba-77db-42a5-a946-98f0c9c2cfd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:23.804409 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56237cba-77db-42a5-a946-98f0c9c2cfd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:23.806177 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-56237cba-77db-42a5-a946-98f0c9c2cfd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:23.817946 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56237cba-77db-42a5-a946-98f0c9c2cfd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:23.819393 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56237cba-77db-42a5-a946-98f0c9c2cfd7 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:23.828452 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1443/2898] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:26.793084 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4274d-cdd9-433d-b354-b1c889085af1 service neutron] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:26.808670 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4274d-cdd9-433d-b354-b1c889085af1 service neutron] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:26.815544 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed12142-ddfc-46f4-a47a-b36660a1eb53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be08c43f0d0c4a8194596e54500b44ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:26.817066 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed12142-ddfc-46f4-a47a-b36660a1eb53 service nova] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:26.817355 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed12142-ddfc-46f4-a47a-b36660a1eb53 service nova] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:26.817640 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed12142-ddfc-46f4-a47a-b36660a1eb53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:26.819452 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-1ed12142-ddfc-46f4-a47a-b36660a1eb53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:26.834588 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed12142-ddfc-46f4-a47a-b36660a1eb53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:26.836556 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed12142-ddfc-46f4-a47a-b36660a1eb53 service nova] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:26.849099 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1456/2899] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 09:29:33.824234 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=6 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:36.844782 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=6 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:37.866853 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56237cba-77db-42a5-a946-98f0c9c2cfd7 service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:37.875491 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56237cba-77db-42a5-a946-98f0c9c2cfd7 service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:37.879207 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6b080-a97e-4b02-8c95-380f08429387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e5ad4e122ae471cad4cf1b0e3111dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:37.879838 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6b080-a97e-4b02-8c95-380f08429387 service neutron] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:37.880120 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6b080-a97e-4b02-8c95-380f08429387 service neutron] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:37.880395 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6b080-a97e-4b02-8c95-380f08429387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:37.882192 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d1c6b080-a97e-4b02-8c95-380f08429387 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:37.943894 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6b080-a97e-4b02-8c95-380f08429387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:37.945360 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6b080-a97e-4b02-8c95-380f08429387 service neutron] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:37.955003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1444/2900] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:41.184473 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed12142-ddfc-46f4-a47a-b36660a1eb53 service nova] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:41.186008 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-1ed12142-ddfc-46f4-a47a-b36660a1eb53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:41.200249 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed12142-ddfc-46f4-a47a-b36660a1eb53 service nova] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:41.205522 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfc7d3-f937-4624-8a77-04ce78fae8ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '066f69a87f5f48c7b20c1e0fcf5c1107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:41.206827 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfc7d3-f937-4624-8a77-04ce78fae8ec service placement] REQUEST_METHOD: `GET` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:41.207110 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfc7d3-f937-4624-8a77-04ce78fae8ec service placement] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:41.207386 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfc7d3-f937-4624-8a77-04ce78fae8ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:41.209135 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-3bbfc7d3-f937-4624-8a77-04ce78fae8ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:41.220738 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbfc7d3-f937-4624-8a77-04ce78fae8ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:41.223075 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbfc7d3-f937-4624-8a77-04ce78fae8ec service placement] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:41.233627 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1457/2901] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:29:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:29:44.128725 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6b080-a97e-4b02-8c95-380f08429387 service neutron] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:44.130412 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-d1c6b080-a97e-4b02-8c95-380f08429387 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:44.144250 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6b080-a97e-4b02-8c95-380f08429387 service neutron] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:44.149013 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8eaa19-5dcb-4767-a03c-f10ae7392ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:44.149995 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8eaa19-5dcb-4767-a03c-f10ae7392ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:44.150343 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8eaa19-5dcb-4767-a03c-f10ae7392ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:44.150735 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8eaa19-5dcb-4767-a03c-f10ae7392ebd admin admin] PATH_INFO: `/v3/users/ae24e34d799e4db292be714fd49579c7` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:44.151994 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8eaa19-5dcb-4767-a03c-f10ae7392ebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae24e34d799e4db292be714fd49579c7)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:44.154094 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8eaa19-5dcb-4767-a03c-f10ae7392ebd admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:44.202926 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bb8eaa19-5dcb-4767-a03c-f10ae7392ebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae24e34d799e4db292be714fd49579c7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:44.215936 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bb8eaa19-5dcb-4767-a03c-f10ae7392ebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae24e34d799e4db292be714fd49579c7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:44.223762 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bb8eaa19-5dcb-4767-a03c-f10ae7392ebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae24e34d799e4db292be714fd49579c7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:44.224480 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bb8eaa19-5dcb-4767-a03c-f10ae7392ebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae24e34d799e4db292be714fd49579c7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:44.233468 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-bb8eaa19-5dcb-4767-a03c-f10ae7392ebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae24e34d799e4db292be714fd49579c7'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:44.240687 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1445/2902] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:29:44 2022] DELETE /identity/v3/users/ae24e34d799e4db292be714fd49579c7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:29:44.414050 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfc7d3-f937-4624-8a77-04ce78fae8ec service placement] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:44.426135 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfc7d3-f937-4624-8a77-04ce78fae8ec service placement] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:44.484588 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa404139-6c15-43dc-a1b4-0c5bd583658e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:44.485455 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa404139-6c15-43dc-a1b4-0c5bd583658e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:44.485742 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa404139-6c15-43dc-a1b4-0c5bd583658e admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:44.486024 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa404139-6c15-43dc-a1b4-0c5bd583658e admin admin] PATH_INFO: `/v3/projects/a93c746bc7844fbfad63dc528ae346f7` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:44.487372 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa404139-6c15-43dc-a1b4-0c5bd583658e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a93c746bc7844fbfad63dc528ae346f7)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:44.489353 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa404139-6c15-43dc-a1b4-0c5bd583658e admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:44.535063 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-aa404139-6c15-43dc-a1b4-0c5bd583658e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a93c746bc7844fbfad63dc528ae346f7', 'request_id': 'req-aa404139-6c15-43dc-a1b4-0c5bd583658e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:44.540846 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-aa404139-6c15-43dc-a1b4-0c5bd583658e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a93c746bc7844fbfad63dc528ae346f7', 'request_id': 'req-aa404139-6c15-43dc-a1b4-0c5bd583658e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:44.547130 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-aa404139-6c15-43dc-a1b4-0c5bd583658e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a93c746bc7844fbfad63dc528ae346f7', 'request_id': 'req-aa404139-6c15-43dc-a1b4-0c5bd583658e'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:44.559948 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-aa404139-6c15-43dc-a1b4-0c5bd583658e admin admin] The token cache is being invalidate because project a93c746bc7844fbfad63dc528ae346f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:29:44.560276 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-aa404139-6c15-43dc-a1b4-0c5bd583658e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:44.560900 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-aa404139-6c15-43dc-a1b4-0c5bd583658e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:44.562430 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1458/2903] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:29:44 2022] DELETE /identity/v3/projects/a93c746bc7844fbfad63dc528ae346f7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:29:44.567241 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8eaa19-5dcb-4767-a03c-f10ae7392ebd admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:44.568665 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-bb8eaa19-5dcb-4767-a03c-f10ae7392ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:44.582405 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8eaa19-5dcb-4767-a03c-f10ae7392ebd admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:44.639003 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4bd7a6-0d4d-41ce-82a9-836223fa8a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:44.639881 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bd7a6-0d4d-41ce-82a9-836223fa8a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:44.640166 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bd7a6-0d4d-41ce-82a9-836223fa8a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:44.640443 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bd7a6-0d4d-41ce-82a9-836223fa8a53 admin admin] PATH_INFO: `/v3/users/1fdbb89cb94b465a9ca74a3a4055206f` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:44.641641 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4bd7a6-0d4d-41ce-82a9-836223fa8a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fdbb89cb94b465a9ca74a3a4055206f)` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:44.643647 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4bd7a6-0d4d-41ce-82a9-836223fa8a53 admin admin] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:44.649254 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:44.692100 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2c4bd7a6-0d4d-41ce-82a9-836223fa8a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fdbb89cb94b465a9ca74a3a4055206f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:44.704124 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2c4bd7a6-0d4d-41ce-82a9-836223fa8a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fdbb89cb94b465a9ca74a3a4055206f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:44.711116 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2c4bd7a6-0d4d-41ce-82a9-836223fa8a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fdbb89cb94b465a9ca74a3a4055206f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:44.711789 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2c4bd7a6-0d4d-41ce-82a9-836223fa8a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fdbb89cb94b465a9ca74a3a4055206f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:44.718651 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.notifications [None req-2c4bd7a6-0d4d-41ce-82a9-836223fa8a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fdbb89cb94b465a9ca74a3a4055206f'} {{(pid=80334) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:44.724561 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1446/2904] 10.222.0.44 () {58 vars in 1223 bytes} [Wed Jul 20 09:29:44 2022] DELETE /identity/v3/users/1fdbb89cb94b465a9ca74a3a4055206f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 09:29:44.898954 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa404139-6c15-43dc-a1b4-0c5bd583658e admin admin] Authenticating user token {{(pid=80333) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:29:44.900389 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.fernet_utils [None req-aa404139-6c15-43dc-a1b4-0c5bd583658e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80333) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:29:44.913986 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa404139-6c15-43dc-a1b4-0c5bd583658e admin admin] Validating token access rules against request {{(pid=80333) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:29:44.970408 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337ea10-4d4d-4054-b1e9-11e5bffe4b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd01511dd98c7416fb8387a8557165556', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5e32bdaa14a4d31baa44b436b02a242', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80333) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:29:44.971881 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337ea10-4d4d-4054-b1e9-11e5bffe4b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:29:44.972224 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337ea10-4d4d-4054-b1e9-11e5bffe4b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:29:44.972488 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337ea10-4d4d-4054-b1e9-11e5bffe4b94 admin admin] PATH_INFO: `/v3/projects/239abe6695734d3b977851f9e3721225` {{(pid=80333) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:29:44.973668 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337ea10-4d4d-4054-b1e9-11e5bffe4b94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=239abe6695734d3b977851f9e3721225)` {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:29:44.975607 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337ea10-4d4d-4054-b1e9-11e5bffe4b94 admin admin] RBAC: Authorization granted {{(pid=80333) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:29:44.996335 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:45.022938 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7337ea10-4d4d-4054-b1e9-11e5bffe4b94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '239abe6695734d3b977851f9e3721225', 'request_id': 'req-7337ea10-4d4d-4054-b1e9-11e5bffe4b94'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:45.028369 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7337ea10-4d4d-4054-b1e9-11e5bffe4b94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '239abe6695734d3b977851f9e3721225', 'request_id': 'req-7337ea10-4d4d-4054-b1e9-11e5bffe4b94'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:45.035182 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7337ea10-4d4d-4054-b1e9-11e5bffe4b94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '239abe6695734d3b977851f9e3721225', 'request_id': 'req-7337ea10-4d4d-4054-b1e9-11e5bffe4b94'} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:45.047611 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7337ea10-4d4d-4054-b1e9-11e5bffe4b94 admin admin] The token cache is being invalidate because project 239abe6695734d3b977851f9e3721225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80333) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 09:29:45.047930 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7337ea10-4d4d-4054-b1e9-11e5bffe4b94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:45.048559 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG keystone.notifications [None req-7337ea10-4d4d-4054-b1e9-11e5bffe4b94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80333) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 09:29:45.050649 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: [pid: 80333|app: 0|req: 1459/2905] 10.222.0.44 () {58 vars in 1232 bytes} [Wed Jul 20 09:29:44 2022] DELETE /identity/v3/projects/239abe6695734d3b977851f9e3721225 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 09:29:54.721030 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:29:55.044629 dv-cinder-843309-5-iscsi devstack@keystone.service[80333]: DEBUG dbcounter [-] [80333] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80333) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 09:31:53.993582 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4bd7a6-0d4d-41ce-82a9-836223fa8a53 admin admin] Authenticating user token {{(pid=80334) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 09:31:53.995043 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-2c4bd7a6-0d4d-41ce-82a9-836223fa8a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:31:54.134099 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4bd7a6-0d4d-41ce-82a9-836223fa8a53 admin admin] Validating token access rules against request {{(pid=80334) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 09:31:54.192657 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7aa07-ec73-4eab-aa7e-a0cb545de8d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '066f69a87f5f48c7b20c1e0fcf5c1107', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a498284274454d25b1b118a88996e522', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80334) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 09:31:54.193311 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7aa07-ec73-4eab-aa7e-a0cb545de8d9 service placement] REQUEST_METHOD: `GET` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 09:31:54.193602 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7aa07-ec73-4eab-aa7e-a0cb545de8d9 service placement] SCRIPT_NAME: `/identity` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 09:31:54.193883 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7aa07-ec73-4eab-aa7e-a0cb545de8d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80334) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 09:31:54.195666 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.fernet_utils [None req-29c7aa07-ec73-4eab-aa7e-a0cb545de8d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80334) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 09:31:54.208402 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7aa07-ec73-4eab-aa7e-a0cb545de8d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 09:31:54.209989 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7aa07-ec73-4eab-aa7e-a0cb545de8d9 service placement] RBAC: Authorization granted {{(pid=80334) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 09:31:54.219909 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: [pid: 80334|app: 0|req: 1447/2906] 10.222.0.44 () {60 vars in 1365 bytes} [Wed Jul 20 09:31:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 09:32:04.215633 dv-cinder-843309-5-iscsi devstack@keystone.service[80334]: DEBUG dbcounter [-] [80334] Writing DB stats keystone:SELECT=89 {{(pid=80334) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}